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ABSTRACT 

Two areas in the theory of delay systems are studied: structural 
properties and their applications to feedback controls and optimal 
linear and nonlinear estimation. First, we study the concepts of 
controllability, stabilizability , observability and detectability. 

The property of pointwise degeneracy of linear time-invariant delay 
systems is then considered. Necessary and sufficient conditions for 
three dimensional linear systems to be made pointwise degenerate by 
delay feedback are obtained, while sufficient conditions for this to 
be possible are given for higher dimensional linear systems. These 
results are then applied to obtain solvability conditions for the 
minimum time output zeroing control problem by delay feedback. Next, 
we turn our attention to optimal linear and nonlinear estimation. A 
representation theorem is given for conditional moment functionals 
of general nonlinear stochastic delay systems. From this, stochastic 
differential equations are derived for conditional moment functionals 
satisfying certain smoothness properties. We then give a complete 
solution to the estimation problem for general linear delay systems. 
Stability properties of the infinite-time linear optimal control 
system with quadratic cost and the optimal linear filter for systems 
without delays in the observations are studied* I-Jhen appropriate 
structural properties hold, the optimal control system and the optimal 
filter are both sho™ to be asymptotically stable. Finally, the cas- 
cade of the optimal filter and the optimal control system is shown to 
be asymptotically stable as well* 
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CHAPTER 1 


IHTRODUCTION 

1,1 Mathematical Description of PelaY Systems 

The research reported in this thesis deals with certaiu aspects in 
the theory of delay systems. Delay systems constitute a class of 
hereditary systems, dynamical systems whose future behavior depends 
on past events in a fundamental way. Mathematically, the simplest 
delay system can be described by the following differential equation 

xCt) = f[x(t), x(t-T), t] (1.1) 

where x(t) E R*^, and T, a positive constant, is a delay. A moment's 
reflection shows that in order to determine the behavior of the system 
for t 2l one must specify not only x(t^), but the function x(0) on 
the interval [t -T- t ]. If we define the function 

x^: [-T, 0] R^ 

x^(6) = x(t+0) ee[-T, 0] (1.2) 

we see that knowledge of the function x^. is necessary and sufficient 
to determine the behavior of the system for t Thus x^ is the true 

state of the system (1.1), and being an element of some function space 
defined on the interval [-T, 0], it is infinite dimensional. This is 
a fundamental difference between delay systems and finite dimensional 
ordinary differential systems. By a slight abuse of language, we shall 
still call x(t) the "state” of the system (1.1), and call x^ the 
"complete state." 
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Equation (1.1) is often called a differential-difference equation 
of retarded type [1]. From the control theory point of view, we can 
have a controlled delay differential equation 

x(t) = f(x(t), x(t-T), u(t), u(t-T), t] (1.3) 

If the system trajectory is not directly measured, but is observed 
through output variables z(t), one can have systems with delays in the 
observations 

z(t) = h[x(t), x(t-T), t] (1.4) 

In actual applications, a combination of the above basic types of 
equations may, of course, arise. Furthermore, these equations can also 
be generalised to include systems with several delays with time- 
varying delays T(t), or with delays T(x(t)) depending on the state. 

So far, we have described delay systems which have a finite 
number of delays. A more general type of controlled delay systems is 
described by the retarded functional differential equation [2] 

x(t) = f[x^, u(t), t] (1.5) 

where x^ is defined by (1.2), This includes differential equations 
with point delays (1.1) and equations with distributed delays 

f[x(t+6)]d0 + u(t) (1.6) 

While even more general types of delay systems can be defined [3], 
they will not play a role in this thesis and hence will not be discussed. 
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Finally, delay systems may also be subject to random disturbances. 
One formulation of a stochastic delay equation is given by 

dx(t) = f[x(t), xCt-T)^ t]dt + dw(t) (1.7) 

where w(t) is a Wiener process, and the equation is understood in the 
Ito sense. Usually associated with the stochastic delay equation (1.7) 
is an observation process 

dz(t) = h[x(t), x(t-x), t]dt + dv(t) (1.8) 

As in the deterministic case, the function x^ plays an important role 
in the theory of stochastic delay systems. 

1.2 Dynamical Processes Modeled by Delay Systems 

Euler was the first mathematician to study delay differential 
equations [4]. Miile there were a number of mathematical investiga- 
tions of such equations after Euler, notably Volterra [5], the basic 
mathematical foundations for delay systems were established in the 
nineteen forties, fifties and early sixties [1], [6]. Since then, 
the theory of delay systems and, more generally, that of hereditaiTf 
systems have been the subject of intensive investigation. While it 
has long been knotm by control theorists that delay systems are the 
appropriate models for a wide variety of process control systems [7], 
recent applications of system theory have shown that soma biological, 
ecological, economical and social processes also take the form of 
delay systems [8], [9]* This has generated a great deal of Interest 


] 
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In various aspects of the theory of delay systems in the control as 
well as mathematical community* We shall discuss some examples of 
such processes . * 

Example 1* Chemical Control Processes 
A basic configuration in many chemical process control systems is the 
following [10]: 



Recycle loop 


Here u usually represents a control of the feed rate of raw materials, 
T represents a transformation which changes the relative composition 
of the reactants. A typical example in refineries is that T is 
composed of a cascade of a reaction cooler, a decanter, and a dis- 
tillation column. The recycle loop feeds back a certain percentage 
of the raw materials and some byproducts of the chemical reaction. 

This recycling introduces a significant time delay, often of the order 
of a few minutes, into the system. To build a suitable mathematical 
model for the analysis of such chemical control plants, a delay system 
must be used. 

Example 2 . Estimation in Radar-Sonar Problems 
We consider the simplest model of a radar systf»*ia [11]: 


t 





Because of the distance of the target, the received signal is a delayed 
(possibly distorted) version of the transmitted signal. If we assume 
the target is a slowly fluctuating point target, the reflection is 
linear, and there are K interfering targets, the received signal can 
be ’C'Tritten in the form [11] 

K jai.t j£i3 t 

r(t) == >^ Re \ ^ B^?(t-t^)e ^ + n(t)]e ^ } 

xjhere f(t) is the complex envelope of the transmitted signal, the 
energy of the signal, the carrier frequency, n(t) the complex 
representation of an additive Gaussian noise process, and for the i^^ 
target, b^ summarizes the amplitude information, is the round trip 
delay time, and is the Doppler shift frequency. These multiple- 
target situations are often referred to as communication over multi- 
path channels. A basic estimation problem for these radar-sonar 
systems is to find optimal estimates for the quantities and 
i = 0,..., K, since these give useful information about the ranges 
and velocities of the various targets. This is clearly an important • 
application of the estimation theory of stodiastic delay systems. 
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Example 3. Modeling of Epidemics 


Delay differential equations have often been used in the modeling of 
epidemics. Let ^c^Ct), x^Ct), and x^(t) denote the number of 

susceptible but unexposed individuals; infective individuals, removed 
individuals, and exposed but not infective individuals respectively. 
Since the infective individuals are those who were infected some time 
earlier, this introduces delays in the dynamics. The following equa- 
tion has been proposed for the spread of measles [12]; 

Xj^(t) = -3(t)x^(t) [2y + x^(t-14) - x^(t-12)] + y 

where B(t) is a proportionality coefficient that is seasonally 
dependent, y is the (constant) rate at which individuals enter the 
population, and a unit of time is a day. 

A more complex model of controlled epidemics has also been 
proposed [13]. Letting u^^ represent active immunization, U 2 passive 
immunization, the model proposed is 

Xj^(t) = -Bx^(t)x2(t) - Xj^(t)uj^Ct-T) - x^(t)u2(t) 

X2(t) = a(t) - r(t) - X2(t)u2(t) 

X3(t) = r(t) 

x^(t) = k(t)3x^(t)x2(t) 

Here 3 is an "effective contact" rate, a(t) the normalized arrival rate 
of infectives, r(t) the normalized removal rate of infectives, and T is 
a delay time. 


- 11 - 


Many more examples of processes modeled by delay j or more 


generally, hereditary systems can be found in [8] and [9]. 

1.3 Outline of Thesis 

From the above discussion, it is clear that delay systems arise 
in many practical applications. This provides ample justification 
for studying the theory of delay systems in depth. There is a large 
literature on the existence and uniqueness of solutions to delay 
differential equations xri.thout control and their qualitative properties 
[2], [14]. However, from the system theory point of view, a large 
number of system-theoretic concepts remain to be explored and examined. 
In this thesis, we study two areas of interest: structural properties 
of delay systems and their applications, and the estimation of sto- 
chastic delay systems. In the first area, we are interested in seeing 
what are the roles played by the structural properties in control and 
estimation problems, and how properties peculiar to linear delay 
systems can be exploited to solve control problems not handled by 
finite dimensional linear systems. In the second area, we are motivated 
by the desire to develop an estimation theory suitable for multipath 
communication problems such as that described in Example 2 of this 
chapter. Of course, these by no means exhaust the theoretical im- 
plications or the practical applications of the ideas and results 
developed in our work. An outline of the thesis is given below. 

In Chapter 2, we study the concepts of controllability, sta- 
bilizability , observability, and detectability in connection with 
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delay systesis. These are properties of great interest in any study 
of dynamical systems. TJhile they merit an in-depth study in their 
own right, our interest in them stems mainly from their relevance to 
the stability properties of the linear optimal control system with 
quadratic cost and the linear optimal filter. The various notions 
of controllability and observability are therefore examined and 
compared from the standpoint of these two problems. The results in 
this chapter will be used in a crucial way in Chapter 6. 

VJhile Chapter 2 deals with standard system-theoretic concepts, 
in Chapter 3 we study a property which is peculiar to delay systems: 
the notion of pointwise degeneracy . This property has been studied 
previously by various authors [15] - [20] as an interesting aspect 
in the theory of delay systems. Our viewpoint, on the other hand, 
is to apply it to the control of linear systems. This motivates the 
development of a useful characterization of systems which can be made 
pointa?ise degenerate by delay feedback. These results serve as the 
foundation for the construction of delay feedback controllers. 

In Chapter 4, we discuss the potential applications of the 
pointwise degeneracy property in delay feedback control for the 
minimum time output zeroing problem for linear systems. The results 
of Chapter 3 are then applied to obtain conditions under which the 
control problem can be solved. The sensitivity of such a control 
system under perturbations of its parameters is also studied. 
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Starting with Chapter 5, we turn our attention to the filtering 
problem for stochastic delay systems. We first give a general dis- 
cussion and show how previous results are inadequate for solving the 
filtering problem for delay systems. We next give a representation 
theorem for conditional moment functionals of nonlinear stochastic 
delay systems. Under suitable conditions, stochastic differential 
equations for the conditional moment functionals can be derived from 
the representation theorem. We then specialize these resiilts^ to 
linear systems and give a complete solution to the filtering problem 
in this case. The similarities as well as differences between the 
optimal linear filter for delay systems and that for ordinary dif- 
ferential systems are discussed. 

In Chapter 6, we study the stability properties of the linear 
optimal delay system with quadratic cost and the linear optimal 
filter. When appropriate system- theoretic properties of control- 
lability, stabilizability, observability, and detectability hold, 
asymptotic stability of these optimal systems can be established. 

The striking duality between control and estimation is also demon- 
strated, This gives what we believe is the first instance other 
than the ordinary linear differential systems case where the control 
and the filtering problems can both be satisfactorily solved, 

Finally, we summarize our findings in Chapter 7 and suggest 
some directions for future research. We shall see that many more 
theoretical problems associated with delay systems remain to be 



reported in tills thesis will 
a full-fledged theory for 
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CHAPTER 2 


STRUCTURAL PROPERTIES OF LINEAR TIME-INVARIANT DELAY SYSTEMS 
2.1 Introduction 

In this chapter* we study four of the basic structural properties 
of linear time-invariant delay systems; controllability* stabilizability , 
observability, and detectability. In contrast to finite dimensional 
linear time-invariant systems, where simple necessary and sufficient 
for these properties to hold are known [21], [22], the situation in 
delay systems is considerably more complicated. For example, there is 
more than one meaningful notion of controllability or observability, 
and the one that is more appropriate depends on the application we have 
in mind. The property that has been studied most extensively is that 
of pointwise controllability [23] - [28] (also known as relative con- 
trollability or Euclidean space controllability). Necessary and 
sufficient conditions have been given for this property to hold, and 
various algebraic criteria have been devised [23] - [28]. Stabiliza- 
bility and functional controllability have also been studied by some 
authors [23], [25], [29] - [34], but the results are far from complete. 
On the other hand, observability and detectability have hardly been 
touched upon in the literature [29]. Our motivation in studying 
these properties is to see how they can be utilized to solve the 
stability problems for the linear optimal control system and linear 
optimal filter (see Chapter 6), The appropriate notions of con- 
trollability and observability for these problems turn out to be 
different from previously given definitions, and we shall explore 
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the relatioiis of these various notions. We make no claim to complete- 
ness or depth of our investigations here. In fact, a great deal of 
work is needed before these structural properties of delay systems 
can be clarified. 

2.2 Controllability 

We shall be concerned with the controllability properties of 
the system 

x(t) = Ax(t) + Bx(t-T) + Cu(t) 

x(0) = (|)(0) 0£[-T,O] (2.1) 

where A, B, C are nxn, nxn, and nioH matrices respectively. We shall 
denote the solution to (2.1) by x(t,^,u). We shall also write x^(<|tju) 
for the complete state of the system (2.1) due to the control u, and 
it is defined by 

x^(4»ju)(0) = x(t+9j^,u) 0e[-T,O]. 

As indicated in Chapter 1, the state space of a delay system is a 
function space. Several choices for the state space are possible. 

The most common one is the space of continuous functions C([-t, 0];R’^) 
equipped with the sup norm which we will abbreviate by ^[2], Other 
choices arc (which is isomorphic to r’^xL^) (29] and W 2 ^^\ the 
Sobolev space of absolutely continuous function on [-T,0] with 
derivatives in L2 [-t, 0] [33]. Each has its own advantages and dis- 
advantages. For a discussion of this, see [35]. In this thesis, 
we will usually take ‘g’ as our state space. Occasionally, we will 
discuss other spaces when ^ proves to be inappropriate for the 
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problem at hand or when other choices shed light on the problem under 
consideration. 

Since the state space is a function space, there are more than 
one notion of controllability that makes sense. The first systematic 
study of controllability for delay systems is that by Kirillova and 
Churakova [23] and is concerned with point^?ise controllability. For 
simplicity, we will take our set of admissible controls U to be all 
R™-valued piecewise continuous functions on some interval [0,T], which 
we shall denote by PG([0,T]; R^) . The same basic arguments can be 
used for U = L2([0,T]; R™) , the space of R”*-valued square integrable 
functions on [0,T]. 

Definition 2.1 The delay system (2.1) is called pointwise controllable 
if for any 4> and any e R.^, there exists a time T and an admissible 
control u on [0,T] such that x(T,^,u) = x^. We also speak of point- 
wise controllability on [0,T] or at time T if the terminal time T is 
fixed. 

By the variation of constants formula [2], [36], the solution to 
(2.1) can be written as 

rO 

x(t 3 (f) 3 u) = $ (t^O) (|)C0)“i- I 4>(t,s*H:)B<i)(s)ds+ 

where $(t,s) is the fundamental matrix associated with (2.1) and ■ 

■ ■ i 

satisfies . j 

I 

A 

■. -I 

-18- 










— $(t,s) = A$(tsS) + B®(t-T,s) 

$(s,3) - I 

#(t,s) =0 t < s (2.3) 

We will also write (2.2) as 

x(t, 4 > 5 u) = x(t,(j),0) + F^u 

where for any fixed t > 0, F^: D -4- 

— t 


A straightforward argument shows that the following is true 
[37], [26]. 

Lemma 2.1 System (2.1) is pointwise controllable if and only if 
there exists a time T such that^(F^) = where ^(X) of an operator 
X denotes the range. An equivalent condition is 

®(T,s)CC'#’ (T,s)ds = n (2.4) 

In contrast to finite dimensional linear differential systems, 
the fundamental matrix $(t,s) defined by (2.3) can be singular [17], 
[35]. Furthermore, pointwise controllability to any point in R°' is 
not equivalent to pointwise controllability to the null vector in R^. 
To distinguish these cases, we make the following 

Definition 2.2 The delay system (2.1) is called pointwise null 
controllable if for any <j>, there exists a time T and an admissible 



/ 


4(t,s)Cu(s)ds 
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control u on [0,T] such that x(T,0,u) = 0. 

It is clear that point^fise controllability implies pointwise 
null controllability. The converse, however, is not true. To 
discuss necessary and sufficient conditions for pointwise null 
controllability, we need the notions of pointwise completeness 
and pointwise degeneracy. 

Definition 2.3 System (2.1) is said to be pointwise complete if 

t 

for each t, there exists a set of initial functions i = 1, . . . ,n 

such that the vectors x(t,4i^,0) i = l,...n form a basis for E?^. If 
the system is not pointwise complete, i.e., if there exists a proper 
subspace V of such that at some time t, x(t,(fi,0) c V for all 
^ the system is said to be pointwise degenerate (at time t). 

The relevance of these notions is shown by (see [26]) 

Lemma 2.2 If the system (2.1) is pointwise complete, then the 
condition 

rank I $(T,s)CC'$(T,s)ds = n 
0 

is necessary and sufficient for pointwise null controllability at 
time T. 

The point is that if the system is pointwise degenerate, we do 
not need ^(F^) = for pointwise null controllability at T. We 
only need^(F,j,) 3 {x(T,<f),0); ^ e^, which is a proper subspace of 
R^ . The notion of pointwise degeneracy has other useful applications. 
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1 


and we shall be studying it in much more depth iji Chapters 3 and 4. 

The condition (2.4) can be replaced by a number of algebraic 
criteria [23], [26], [28]. These can be thought of as generalizations 
of the well-knoim controllability results for finite dimensional 
linear systems. While pointwise controllability is useful in linear 
control problems with target sets in Euclidean space [35], [38], it 
is too weak for other problems. For control problems with target 
sets in function space, we need a much stronger notion of control- 

( 1 ) 

lability. Here, following Banks et al., [33], we adopt the space W 2 
as our state space and as our set of admissible controls U. The 
reason for switching the state space to is that for u £ 1^ 

and <|> e eq. (2.1) implies that x(t) £ L 2 . Hence ¥^ 2 ^^^ ^ 

natural choice for the state space when the control space is L 2 * 

For further discussions on this point, see [33]. 

Definition 2.4 The system (2.1) is said to be functionally con- 
trollable if for any functions (}) and i|; e W 2 ^^\ there exists a time 
T and a control u e L2[0,T] such that x^((]),u) = 1 /;. If the terminal 
time T is fixed, we say that the system is functionally controllable 
on [0,T]. 

The following result of Banks, et al., [33] sho\^s that functional 
controllability on [0,T] where T > T imposes extremely stringent con- 
ditions on the system which are rarely met in practice. 

Lemma 2.3 The system (2.1) is functionally controllable on [0,T] 
where T > T if and only if rank C = n. 
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Functional controllability is too strong for many control 
problems of interest, for example, the stability properties of 
linear optimal feedback systems and linear optimal filters ^ We 
now develop a new notion of controllability x-rtiich is applicable 
to these problems. We revert to ^ and piecex^ise continuous functions 
as our state space and control space. 


Let us first define the function 


?C([0,T]; ->• C([T-t,T]; 


by 


f 


CW^v) (t) = I $Ct,s)v(s)ds t e [T'-T,T] 


Similarly, define 


W^C; U(= PC([0,T]; R™)) C([T-t,T]; 


by 


f 


(W^Cu)(t) = I #Ct,s)Cu(s)ds te[T-x,T] 


Let us denote the space of all R^-valued functions ijj on [t,t+x], 

whi&h are of bounded variation on [t,t+T] and continuous from the 

left on (t,t+r), by B^([t,t+T]; B?^). We equip B^([t,t+r]; R^) 

xvrith the total variation norm on [t,t+T], denoted by Var . For 

[t,t+T] 

e B^([t,t+r]; R^), 4* e C([t-T,t]; r”), define the bilinear form 


<ij>,4)>^ by (see [2]) 




r r 


(s4r)B4>Cs)ds 


(2.5) 
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We shall refer to as the hereditary product at time t. 

t 

We adopt the following 

Defin-rtion 2.5 The system (2.1) is said to be controllable on 
[0,T] if there exists no e R“) such that “ 0 

for all ^ e ^(W^C) , but that ^ 0, for some e ^(W^) . 

This definition of controllability may not be very well- 
motivated at the moment since PC([0,T); E^) is not the space of con- 
trols. It is obtained strictly as the dual notion of observability, 
which we T^ill discuss in section 2.4. The physical meaning of^(W^C) 
is clear and corresponds to functions which can be attained at T by 
suitable control. On the other hand, it is not clear to what^(W^) 
corresponds. However, this definition of controllability is easily 
seen to be stronger than pointtJise controllability and weaker than 
functional controllability. For if we take t = T, and take to be 
such that ijj(T) ^ 0, t|j(s) = 0, T < s £ T + x, x^e get that 


= ti^’(T)^(T) 


Controllability then says that there exists no t{)(T) such that 
^T 

4»’(T) I ®(T,s)v(s)ds 0, for some v e PC([0,T]; R ), but that 

ip' (T) $(T,s)Cu(s)ds = 0, 

*0 

for all u E PC([0,T]j R*^). Since ®(T,s) = for s e [T-r,T], 

for arbitrary e we can define 


v(t) = e^' 


= 0 


T-T 


te[o,T-T) 
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. ..4-- 


rT 

This yields I $(T,s)vCs)ds = and shows that { I f(Tjs)v(s)ds: 

*0 0 

V e PC([0,1]; H?^)} = E.’^. If { I $(T,s)Cu(s)dsJ u e PC([0,T]; B®)} 

*0 

is a proper subspace of then we can find i/>(T) such that 

if(’(T)r #(Tjs)Gu(s)ds = 0, 

*{) 

-T 

for all u e PC([0,T]; R™) . CJhoosing v such that | ®(T,s)v(s)ds = 

•'o 

we see that system (2.1) will not be controllable. Hence controllability 
implies ^(F^) = R° which is precisely pointwise controllability. On the 


other handj if rank C = n, then^Cff^) is clearly the same as ^(W^C) , from 
which controllability follows. Hence functional controllaibility implies 
controllability . 

Finally, there is a fourth notion of controllability, that of 
approximate functional controllability. 

Definition 2.6 System (2.1) is called approximately functionally 
controllable if for any ^ and e there exists a time T and a 
sequence in U such that x^(<f>,u^) converges to i/). 

Again approximate functional controllability is weaker than 
functional controllability, but stronger than pointwise controllability. 
Its relation to our definition of controllability is not known at 
present, its use in delay system problems seems to be rather limited, . 
However, it is one of the standard notions of controllability in in- 
finite dimensional linear systems [39], and we have stated it for 
completeness. 
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2.3 Stabflizability 

Wo start with the definitions of stability and assrmptotic stability 
of unforced delay systems. 

Consider the system 

x(t) = Ax(t) + Bx(t-T) 

x(6) = (j)(0) 0 e[-T,0] (2.6) 

where <p £ 

We shall denote the solution to (2.6) by x(t,0) and the complete state 
of (2.6) by (({)). 

Definition 2.7 System (2.6) is said to be stable if for every e > 0, 
there exists a 6(e) > 0 such that <6 implies the solution of (2.6) 

satisfies jx^] < e, for all t ^ 0. 

Definition 2.8 System (2.6) is said to be asymptotically stable if 
it is stable, and there is a 6^ such that j<^j implies the 

solution of (2.6) satisfies lim| x (<j>) | =0. 

It is kno^TO. that asymptotic stability of linear delay systems is 
equivalent to exponential stability [2], [36]. In other words, (2.6) 
is asymptotically stable if and only if there are positive constants 
K and a such that |x^(4>)| ^ Ke t ^ 0, for all ^ e 

We are now ready to give the definition of stabilizability. 
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Definition 2.9 System (2.1) is said to be stabilizable if there 


exists a matrix function L; [— T»0] of bounded variation on 

[—T,0]j such that the closed— loop system 

.0 

x(t) = Ax(t) +Bx(t-T) + C I dL(6)x(t+e) (2.7) 

is asymptotically stable. We also say (A,B,C) is stabilizable to mean 
this definition. 

As in the control of finite dimensional linear systems, stabiliz- 
ability is crucial for the well-posedness of infinite-time control 
problems for delay systems (see Chapter 6) . Necessary and sufficient 
conditions for stabilizability can be given in terms of controllability 
of a finite <^imensional system obtained by projecting the complete 
state x^ onto the generalized eigenspaces associated with the eigen- 
values with nonnegative real parts [30], [31]. This result is 
difficult to use, however, because it is necessary to compute the 
eigenvalues and find a basis for the generalized eigenspaces assoc- 
iated with them. One would like to have conditions expressed 
explicitly in terras of the system parameters A, B and C. Since the 
decomposition of the complete state x^ has been given in detail by 
Hale [2], this appears to be a hopeful, but still open task. 

In finite dimensional systems, controllability is a sufficient 
condition for stabilizability. Since the controllability condition 
is very easy to check, this is an extremely useful sufficient con- 
dition. It is important to see, therefore, if some notion of con- 
trollability implies stabilizability in infinite dimensional linear 
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systems. This is an aspect of the theory of infinite dimensional 


linear systems that has received quite a bit of attention recently 
[40], [41], We have seen in the above that for delay systems, there 
is indeed a notion of controllability that is necessary and suf- 
ficient for stabilizability , This notion, however, does not cor- 
respond to any of those discussed in section 2-2* It is worthwhile 
to give a few remarks concerning the relationships between those 
notions and that of stabilizability. 

It is clear that functional controllability is sufficient for 
stabilizability, since functional controllability enables us to move 
to the zero function in finite time. Approximate functional con- 
trollability is also sufficient since this certainly implies that 
finite dimensional projections are controllable. On the other hand, 
pointwise controllability is not sufficient for stabilizability. 

An example illustrating this has been given by Morse [42]. The 
relationship between controllability in the sense of Definition 
2*5 and stabilizability is not known. 

2.4 Observability 

Here we are concerned with the system 
x(t) = Ax(t) Bx(t-T) 

x(9) = (f)(e) 9 e[-T,0] (2.8 

. z(t) = Cx(t) t ^ 0 (2.9 

We would like to know when it is possible to deduce the trajectory of 
the system x(t), 0 ^ t T, from the observations s(t) , 0 ^ t £ T. 



In contrast to controllability, pointwise obseirvability has very 
little meaning. This is because knowledge of x(a) is in general 
not sufficient in determining the evolution of the system for 
t ^ (J. We shall define two notions of observability. 


Definition 2.10 The system (2.8) - (2.9) is said to be observable 
if for any (j), z(t) =0, t 0, implies x(t) = 0, t ^ 0. It is 
observable on [0,T] if z(t) = 0, 0 t ^ T, implies x(t) = 0, 

0 £ t < T. 

Since (2.8) - (2.9) are time-invariant, an equivalent definition 
is; z(t) =0, t £ s, implies x(t) =0, t £ s, any s £ 0. It is useful 
to rephrase Definition 2.10 in terms of the initial function ((). 

Definition 2.10 * The system (2.8) - (2.9) is said to be obsesrvable 
if there exists no function B such that z(t) = 0, t £ 0, and such 
that x(t) is not identically zero for t £ 0. 


Let us express Definition 2.10* in terras of certain operators. 
By the variation of constants formula 


z(t) = C$(t,0)<|)(0) + J C#(t,s+r)B(!)(s)ds (2.10) 

“T 

Define the operator PC([0,T]; E?^) by 

(M^(()) (t) = ®(t, 0)^(0) + I $(t,s+T)B<i)(s)ds (2,11) 

-T 

for any t e [0,T]. Similarly, define PC([0,T]; II"^) by 

(Cli|,4D(t) = C0(t,O)(f)(O) + I C$(t,s+T)B{|)(s)ds (2.12) 

"T 
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I 



for any t e [0,T], Then the system (2.8) - (2.9) is observable on 
[0,T] if and only if 

K(iyip) = N(CM^) (2.13) 

where N(X) of an operator X denotes the nullspace of X. We mil now 
prove the following duality result which relates the notions of con- 
trollability and observability. 


Theorem 2.1 The delay system 

x(t) = Ax(t) + Bx(t-T) + Cu(t) 


x(0) = 0(0) 0 E[-T,0] (2.14) 

is controllable on [0,Tl if and only if its hereditary adjoint system [2], [36] 

y(t) = -A'y(t) - B'y(t+T) 

y(0) = 0(6) 0 e[T,T+x], ([T,T+X] ; r’^) (2.15) 

z(t) = C’y(t) (2.16) 

is observable on [0,T]. 


Remark We shall usually simply say (2.15) - (2.16) is the adjoint or 
dual system to (2.14). Of course, the adjoint system (2.15) evolves 
backx^ards in time starting at t = T. 


Proof ; Applying the variation of constants formula to the adjoint 
system [2], [36], we obtain, for t £ [0,X] 

y(t) =$*(T,t)0(T) + I (s-T,t)B’0(s)ds 

= (H|0)(t) (2.17) 
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Hence observability of the adjoint system is equivalent to 
N(C’H|) - N(H|) 

Relative to the hereditary product, the adjoint of is given by 
PC([0,T]; r’^) ^ G([T-t,T]; r"^) 

v>p^ = <ij», where <u,v>p^ = j u’(t)vCt)dt. 

•/n 


We calculate 


<H^, v>p^ = f t|)’(T)$(T,t)v(t)dt 

J ^T ^T+T 

I 

0 *^T 


(s)Bf Cs“T,t)v(t)dsdt 


)’(T) r’ 

Jq 


•T rT+r 

l|j’(T) I $(T,t)v(t)dt + I l{j'(s)B I #(s-T,t)v(t)dtds 

“'T •'0 


= <il), H^v>^ 


Hence (H^v) (t) 


/•T 

[ $(t. 


s)v(s)ds te[T-T,T] 


The reader may note here that = W^, with defined in section 2,2. 
Similarly, the adjoint of C’H* is given by 

H^C: PCC[0,T]; r“) -s- C([T-t,T1; r“) 

(Hj,Cv)Ct) = f S(t,s)Cv(s)ds te[T~T,T] 

‘'O 

(Note that H^C = W^G) . Now ij; e K (H*) if and only if 


v>p^ = 0 for all v e PG([0,T]; R^), 
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i.e. , Ip is in the annihilator relative to the hereditary 

product. Hence the condition N(H*) = N(C*H*) says that there exists 
no ijj such that 

= 0 ¥ (p e 

but <i{)j ^ 0 for some {()^ c ^(H^) 

which is precisely the condition required for controllability of the 
original system. 

This duality result mil be used in a crucial way in Chapter 6. 
Since it is based on the use of the hereditary product, it is not 
the same as the functional analytic duality betoeen the various 
operators and spaces involved. It would be interesting to see what 
duality result would be obtained if we use functional analytic ad- 
joints for delay systems, as expounded in Hale [2]. 

Another definition of observability which is stronger than 
Definition 2.10 is that of strong observability. 

Definition 2.11 The system (2.8) - (2.9) is said to be strongly 
observable if z(t) =0, t ^ 0 Implies the initial function ^ = 0. 

Clearly, if the system is strongly observable, it is observable. 

The converse is not true. Indeed, the matrix B must necessarily be 
nonsingular if strong observability is to hold. Otherwise we can 
choose an initial function (j) e*^ such that (j>(0) = 0 and 0 ^ (j>(s) e N(B), 
s e [-T,0). This yields z(t) =0, t ^ 0, but (j) ^ 0. 
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By exactly the same arguments as in Theorem 2,1, we can show that 
the dual of strong ohseanrability is the following notion of control- 
lability: there exists no e B^C[T5T4 t]; R^) such that = 0 

Again B must necessarily be nonsingular. This may be taken as still 
another notion of controllability if desired. 


2.5 Detectability 

As in finite dimensional linear systems ^ this is basically the 
dual notion of stabilizability . 


Definition 2.12 The system (2.8) - (2.9) is said to be detectable 
if there exists a matrix function K: [-TjO] of bounded variation 

on such that 


x(t) ™ Ax(t) + Bx(t-r) ^ 


I dK(0)z(t+9) 

-T 


is as 3 nnptotically stable. 


(2.18) 


Yheorem 2.2 The delay system (2.8) - (2.9) is detectable if and 
only if the adjoint system 

y(t) = *A*y(t) - B*y(fc‘+r) “ C’u(t) (2.19) 

is stahilizable (again this system runs backward in time) . 


Proof ; Stabilizability of (2.19) requires the existence of a matrix 
function K’ : [-T,0] of bounded variation on [-T,0], such that 


-y(t) = A*y(t) + B^y(t-K) + C 


’ p dK’ 

-T 


(0)y(t-0) 


( 2 . 20 ) 


is asymptotically stable. The right hand side of (2.20) defines a 
Stieltjes integral of the form 
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I dn' (6)yCt~9) 

I 

Ic'K' (6) 0 10 

where n^0) = j-A'+C’K’ (6) -T < 0 < 0 

|-A’-B’-C’K’ (0) 9 £ -T 

The hereditary adjoint system to (2.20) is given by [2] 
rO 

x(t) = I driC0)xCt+0) 

•^T 

Substituting for ri(9), we get that (2.21) is 

^0 

x(t) = Ax(t) + Bu(t-T) -5- I dK(0)Cx(t+0) 


= Ax(t) + Bx(t“T) + ! dK(0)z(t-{-0) 


Since the stability properties of the adjoint system are the same as 
those for the original system [2], stability of (2,20) is equivalent 
to stability of (2.22). Hence the theorem follows. 


The remarks following the proof of Theorem 2.1 concerning the use 
of adjoint systems also applies to this case. 


Analogous to the situation for stabilizability, it would be 
interesting to see what notion of observability implies detectability. 
Presumably this would be the dual of that notion of controllability 
which implies stabilizability. 


From the above discussiouj it is clear that a great deal of 
work remains to be done in studying the concepts of controllability, 
stabilizability, observability, and detectability. We have merely 
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formulated definitions which appear to be useful in control and esti- 
mation problems, and pointed out some of the relationships between 
these concepts# Our discussion has not even exhausted the definitions 
of controllability and observability which have appeared in the lit- 
erature# Motivated by the algebraic approach to finite dimensional 
linear systems, Kamen and Morse [42], [43] have given some algebraic 
definitions of controllability and obseir^ability which are useful in 
realization theory for delay systems# It is not clear what the 
physical interpretations of these definitions are, or what their 
relationships to our definitions are. We shall leave this entire 
area as a subject for future research# 


CHAPTER 3 


POINTWISE DEGENERACY OP LINEAR DELAY SYSTEMS 
3.1 Introduction 

In Chapter 2, we studied some structural properties of linear 
delay systems and introduced the concept of pointwise degeneracy. 

In this chapter, we shall study the degeneracy property in much more 
depth, with a view towards applying it to feedback control. The 
degeneracy property of delay systems has recently attracted the 
attention of many researchers [15] - [20], partly because of its 
connections xjith other system-theoretic concepts (see Chapter 2), 
and partly because it is peculiar to delay equations and has no 
counterpart in ordinary differential equations. Popov [15] is the 
first to make a systematic study of the subject. His fundamental 
results have laid the foundations from’ which our results are de- 
veloped. Since the publication of his paper, many other results 
have been obtained on pointwise degeneracy. It is fair to say, 
however, that this peculiar property is still far from being 
completely understood. Our viewpoint differs somewhat from these 
previous investigations in that we are primarily interested in 
applying the property in the construction of delay feedback controls. 
We do not therefore study pointX'/ise degeneracy as an intrinsic 
property of delay systems. Rather, we study conditions under x^'hich 
a linear system can be made pointwise degenerate by delay feedback. 
Since we rely heavily on the techniques developed by Popov, in 
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Section 3.2, we shall first review his restilts and others which are 
germaine to our subsequent development. In Section 3.3, we give a 
characterization of systems which can be made pointwise degenerate 
by delay feedback. We shall then explore some of the implications 
of this characterization. This will prepare the way for the dis- 
cussions in Chapter 4 where these results are used to obtain delay 
feedback controls for certain problems in linear system theory. 

3.2 Some Existing Results on the Pointwise Degeneracy of 
Linear Delay Systems 

We shall be concerned primarily with linear, constant delay 
systems of the form 

x(t) = Ax(t) + Bx(t-T) (3.1) 

with an initial function e We first give an alternative de- 

finition of pointwise degeneracy of Eq, (3.1). 

Definition 3.1 The linear delay system (3.1) is called pointwise 
degenerate if there exist an n-vector q 9 * 0 and a number t^ > 0 
such that every continuous function x: [-T, t^^] bP' satisfying 
(3.1) in the open interval (0, t^^) satisfies also q’x(t^) = 0. 

If the system is not pointwise degenerate, it is called pointwise 
complete . 

Notation. We will abbreviate pointwise degenerate by p.d. and 
pointwise complete by p.c. We will also say (A, B, q, t) is p.d, 
at tj^ or (A, B, t) is p.c. to mean the above definition. If the 


time of pointwise degeneracy is of no conceim, we will drop the 
qualification "at t^^" from our statements. 

Definition 3.1 is readily seen to be equivalent to Definition 
2,3 of pointoise degeneracy. However, Definition 3.1 has proved 
to be much more useful in obtaining results. We will adopt this 
definition of point^^ise degeneracy from now on. 

Let us note the following important consequence of Definition 

3.1: 

If (A, B, q, t) is p.d. at t^^, then it will be p.d. on an interval 
of the form [t^^, «>), which we will call the degeneracy interval. To 
see this, take any number t^ > For any continuous function 

X! [-T, t^] ^ satisfying (3.1) on (0, , define the shifted 

function x: [-T, t^] ^ by x(t) = x(t + tg - . Then x is con- 

tinuous and satisfies (3.1) in (0, t^). Hence q'x(t^) = q*x(t 2 ) = 0 
and X is p.d. at t 2 also. 

This simple fact is of central Importance in the application of 
the p.d. property in delay feedback control, as will be e:q>lained 
later. In fact, a stronger result holds for the structure of the 
degeneracy interval. This will be given in Theorem 3,1. 

Definition 3.1 admits a simple geometric interpretation. For 
each initial function there corresponds a solution of (3.1), 
which we will denote by x(t, (|)) . Then pointwise degeneracy means 
that there is a fixed vector q such that the solutions x(t, <|)) evolve 


in a space perpendicular to q, for any choice of 0 e^. 



The first example of a p.d. system was given by Popov [15], 
which has 



B = 1 0 0 

[o 2 oj 

qt = IX _2 _i] 

and T = 1 with a degeneracy intearval of [2, «>) . This example sparked 
a surge of interest in the p.d. property. By now, much more is known 
about this example. We shall retuam. to it in Chapter 4. 



Most of the basic results concerning pointwise degeneracy have 
been given by Popov. The following theorems are fundamental. 


Theorem 3.1 (Popov [15]). Assume that (3.1) is p.d. for the vector 
q. Then the largest set of points t^^ at which (3.1) is p.d. for q 
is the interval []^T, “), where Z is the smallest integer with the 
property 

q»S(a)^'*'^ = 0 (3.2) 

and S (ct) is the nxn polynomial matrix given by 
(al-A)S(cj) = B det(al-A) 


Moreover, 2 £ 2 < n~l 


Theorem 3.2 (Popov [15]). The delay system (3.1) is p.d. for q at 
time t^ > 0 if and only if there exist an integer m > 0, k matrices 
P^, mxn (k is the largest integer such that kx ^ t^^) , an mxm matrix 
V, and an m-vector v such that 


Pj^B = 0 


P.A + P. .,B = VP. 
3 J+1 3 


V = ^k 


v'e^'^^P^ « 0 


, VT 

v’e P.,,-v'P. = 0 
J+1 3 


v'P, = q’ 


(3.3) 

j = l,2,..k-l (3.4) 

(3.5) 
, (3.6) 

j = 1.2...,k-l (3.7) 

(3.8) 
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Moreover, if the above quantities exist, one can always choose them 
so that rank = m. 

From these theorems, one can deduce the following corollaries, 
some of which were obtained by other authors. 

Corollary 3.1 (Popov [15]). System (3.1) is p.c. if rank B = 1. 

Corollary 3.2 . System (3.1) is p.c. if rank B = n. 

Proof : If rank B = n, (3.3) implies Pj^ = 0. Using (3.4) repeatedly, 
we immediately see that P 2 > . . . , are all equal to zero, and the 
corollary follows. 

This result was apparently established earlier by E.B. Lee. 
Corollary 3. 3 . System (3.1) is p.c. if n ^ 2. 

Proof ; If n = 1, Corollary 3.1 immediately shows that (3.1) is p.c. 
Assume therefore n = 2. In this case, rank B is either 1 or 2. But 
then CorollaiT^ 3.1 and 3.2 together imply (3.1) is p.c. 

This result was apparently established earlier by J.A, Yorke and 
J. Kato. 

Although in the proof of Theorem 3.2, Popov gave’ a constructive 
procedure for finding the matrices P^^, . . . , P^, V, and the vector v, 
it is so complicated that in applications. Theorem 3.2 will be difficult 
to use. However, Popov was able to isolate a class of p.d. systems 
which have a remarkably simple structure. These are the so-called 


regular pointwise degenerate (r.p.d.) systems . 


Defipition 3.2 . A p.d. system of the form (3.1) is called "regular” 
if the pair (A, B) is completely controllable and there exists an 
n-vector q such that the pair (q’. A) is completely observable, and 
(3.1) is p.d. for q in the interval [2 t, “) . 

For such systems, the following theorem is true. 

Theorem 3,3 (Popov [15]). Suppose (3.1) is r.p.d. with respect to q. 
Then there exists an nxn matrix Z such that 


ZAZ = Z^A 

(3.9) 

q'Z^ = 0 

(3.10) 

q'Z = q'e^"^ 

(3.11) 


Equation (3.1) then takes the form 

x(t) = Ax(t) + (AZ-ZA)x(t-T) (3.12) 

(that is, AZ-ZA = B). Conversely, every equation of the form (3.12), 
in which Z satisfies (3.9) - (3.11), is p.d. for q in [2x, ") (even 
though the other conditions for regularity may not be satisfied). 

The following corollary can be deduced from Theorem 3.3. 

Corollary 3.4 . (Popov [15]). If B can be written as AZ-ZA where Z 
is given by 

At 

z = rq’e""'- (3.13) 

and r is the solution to the equations 
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1 




q'r = 1 

(3.14) 

11 

o 

(3.15) 

q’e^^Ar = 0 

(3.16) 


then (A,B,q,T) is p.d. on [2x, «») . 

Finally, Popov [15] has proved the following important result 
for 3 dimensional p.d. systems. 

Theorem 3.4 . Any 3 dimensional p.d. system is regular and can be 
written in the form. (3.12) with Z given by (3.13) - (3.16). 

For later reference, we emphasize here that our results on 
pointwise degeneracy and delay feedback are heavily based on Corollary 
3.4 and Theorem 3.4. It is also worth pointing out that while (3.13) - 
(3.16) are considerably simpler than (3.3) - (3.8), they still involve 
the calculation of matrix exponentials. Given a triple (A,q,x) it 
is not obvious from (3.14) to (3.16) when a solution for r will exist. 
Our results given in section 3.3 will be concerned precisely with 
simple conditions on (A,q,x) under which a solution r to (3.14) - 
(3.16) exists. 

Some additional properties of r.p.d. systems can be deduced from 
(3.9) - (3.12). For example, the Z matrix is in fact unique for 
regular p.d. systems. To see this, suppose and Z^ both satisfy 
(3.9) - (3.12). Then 

q’(Zj - Z^) = 0 
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q'&Z^ = q'(Z^A + B) = q’Z^A + q’B 
= q’e^^A + q’B 

this Implies 

q»A(Z^ - Z^) = 0 

Similarly, it can be shown that 

q'A^(Z^ - Z^) = 0 i = 0,...,n-l 

By observability of (q'. A), and uniqueness follows. It can 

n— 1 n 

also be shown that Z = 0 and B = 0 for r.p.d. systems. The sig- 
nificance of these properties are not known at the present time and 
we shall not pursue them further here. 

We have summarized in the above some known results on pointwise 
degeneracy. We will apply these in our investigations in the next 
section. 

3. 3 A Criterron for Pointwise Degeneracy for Linear Delay Systems 

Let us motivate our investigations by the following control problem 
(see also Popov [44]). Consider the linear control system 

x(t) = Ax(t) + u(t) (3.17) 

y(t) = q'x(t) (3.18) 

The objective is to find a linear state feedback law, possibly delayed 
with a fixed delay time x > 0, to drive the output y(t) to zero in 


I 
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TOinimnTn time for all initial conditions, remaining zero thereafter. 
We shall call this minimum time output zeroing hy delay feedback. 

It is obvious that this problem cannot be solved by using in- 
stantaneous state feedback 

u(t) = Kx(t) 

However, if we can find a matrix B such that (A,B,q,T) is p.d. at 
2t, then by definition of the p.d. property, y(t) = 0 for t ^ 2t. 
Furthermore, since 2 t is the smallest instant at which a delay 
system can be p.d. (see Theorem 3.1), this matrix B will be the 
solution to our minimum time control problem. 

The first question to be examined concerning this approach is 
to find conditions on the matrix A, the vector q, and the delay time 

T, such that a matrix B will exist with the desired properties. 

Corollary 3.4 tells us that if we can find a solution r to (3.14) - 
(3.16), then on constructing the Z matrix as in (3.13) and setting 
B = AZ - ZA, we will obtain a p.d. system with degeneracy interval 
[2T, w). Furthermore, Theorem 3.4 shows that for 3 dimensional 
systems, the above construction gives the unique B. Therefore, for 
3 dimensional systems an equivalent problem is to find conditions 
on A, q, and T such that (3.14) - (3.16) admit a solution. We will 
exploit this in the proof of the following theorem. 

Theorem 3.5 . Given a 3x3 matrix A with eigenvalues X^, a 

3-vector q, and a nuiriber T > 0, a matrix B with the property that 
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(A,B,q,x) is p.d. exists if and only if the following conditions 
are satisfied: 

(i) The geometric multiplicity [45] of each distinct eigenvalue 
of A is one; 

(ii) If there is a pair of complex eigenvalues, say Xg = ^ 3*9 
real, then 

5 + e^^(T}sin^T - ^cos^t) 0 
where n = ^2 " ^ “ - 1 ^X 2 ; 

(iii) (q’,A) is observable. 

Before proving Theorem 3.5, we first state a lemma which is 
completely obvious. This lemma is not restricted to 3 dimensions. 

Lemma 3.1. Suppose (A,B,q,x) is p.d. then for any nonsingular matrix 
F, (FAF FBF F'~^q, x) is also p.d. 

Lemma 3.1 implies that if we are free to choose B such that 
(A,B,q,x) is p.d., there is no loss of generality in assuming A 
to be in Jordan form. Me exploit this in 

Proof of Theorem 3.5 ; Let q’ = [q^^ q 2 q^J* We will apply the p.d. 
criterion established in Theorem 3.4. By the above remarks, we may 
assume, without loss of generality, that A is in Jordan foinn. Since 
n=3, the possible Jordan forms are as follows: 


t 


\ 0 0 

0 X 0 

0 0 X 


X^’s may not be distinct, and there can only be a pair of complex 


eigenvalues. 


X^ 1 0 

0 0 
0 0 X, 


x^ 0 0 

0X^1 
0 0 X, 


^1* ^2 distinct, but they are necessarily real. 


(Ill) 


0 0 X 


X is necessarily real. 


We first note that if there exists a solution x to (3.14) - (3.16), 

At Ax 

the vectors q’, q’e , and q’e A are necessarily linearly independent 
(cf. Popov [15], p. 559). Theorem 3.4 then requires that 


det q ’ 

q'e'^'^J 


» det W iff Q 
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We consider each of these cases separately. 
Case I. We require 

'll ‘I 2 


det W = det 






qse 




^ 1 ^ *^ 2 ® 

XlT 12”^ ^3^ 

‘^ 2 ^ 2 ® ^3^3® 


(X 2 +X 2 )t 


(X^+X 3 )t 


(Xj^+X 2 )t 




Clearly if any = 0, then det W = 0 and (3.1) cannot possibly be 

degenerate for this q. We must also not have 

(X„+X,)T (Xi+X.)T (X-+X„)T 

(X3-X2)e ^ +(Xj^-X3)e ^ +(X2“Xj^)e ^ =0 


(3,20) 

(^2**‘^3^^ 

Dividing throu^out by e i (3.20) becomes 

(X_-X_)T (X.-X_)T 

(X3-X2) + (^i“^3)e ^ ^ =0 (3.21) 


Let = ^X~^2’ ^2 ~ 

Y,T 

f(Yj^,Y2) = (Yi“Y2) + Y 26 "Yx® (3,22) 


Our problem now is reduced to finding conditions under which f(Y^jY 2 ) 0. 

We first consider the case where all the X^'s are real (so that Yj^ Y 2 

are also real) . Without loss of generality, we may assume Yx ^ ^ 

(i.e., ^^ 3 )* Y^ “ 0, we have immediately f(Y 2 >Y 2 ^ “ Suppose 

y T Y T 

Yj^ > 0 is fixed. We plot Y 2 (® ^ Yj^(s ^ ~1) as functions of Y 2 * 

f(Yj^,Y 2 ) = 0 at the points where these two functions are equal, Note 
that at Y 2 ” both functions are zero, and at Y 2 ~ Yx> they are equal. 


- 47 - 


a 



{Y2(e ^ “1)1 = e 1 “1 


and ^ [Y^Ce 2 -i) ] = Yi^e 2 

Y T Y 

In the region where Y 2 ^ 0, Yq^^® ^ 5, < e ^ -1, for Yj^ ^ 0 

this region, therefore 

Y2(e^l^-1) < Yj^Ce^^'^-l) 

We can now see that the graphs of the tT *70 functions look like 






The functions can only intersect at two points. But we already know 
that they do intersect at Y 2 “ 0 "Y 2 “ Hence these are the 

only two points at which they can be equal. So in the region > 0, 
f(Yj^»Y 2 ) = 0 if and only if Y 2 ” Y 2 = 0* conclude 

that if all the are real, f(Yj^,Y 2 ^ = 0 if and only if y^^ = 0, 

or Y^ = 0, Yj^ = Y 2 * other words, det W = 0 if and only if 
either ^2 ^3’ ^3 ^ ^1’ ^i^^ ^ 

Next we consider the case when there is a pair of complex 
eigenvalues, say,X^ = X^*. This implies X^, X„j and X^ are neces- 
sarily distinct. With Yj^ and. Y 2 defined as before, we have that 
Y^ = Y 2 *' Define n = X^^ - HeX 2 > 5 = - 110 X 2 (i-e., Y]l “ ’ 

Then 

f(Yi,Y2) = 2i? + 

= 2i? + e^^^{ (n-iC) (cos?T + isin^T) - (H+iC) (cos^T- isin?T)} 

= 2i[5+e^'^(nsineT - ?cos?T)] (3.23) 

f(Y-|^aY 2 ) “ 0 if and only if 5+®^^ (Hsin^T- ^cos^t) = 0. Thus, if there is 
a pair of complex eigenvalues, we require precisely condition (ii) of 
Theorem 3.5, in addition to no q^ = 0. This completes the consideration 
of Case I. 

Case II. Since the same arguments hold for either 



Xj^ 1 0 


0 

0 

I 

J = 

0 Xf 0 

or 

0 ^2 1 


_0 0 X2_ 


0 0 ^2 
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we only discuss the former situation. In this case, 

“1 


JT 


XlT Xl T 
e xe 0 


XlT 

e 0 


e j 


Thus 


det W = det 






A,t 


AiT 


|q^A^e ® ^3^2 


V , ^ 2 '' 

qoA„e 


= qj^^q 3 e ^ [ (A 2 T- 1 -A^x)e ^ +e ] 


after some calculations. Hence 


det W 0 if and only if # 0, q^ 0 


A,t A.t 

and (A 2 T-l“A^T)e +e 0. 


Let Y = A^ “ A 2 . Then 


A„t A,t 

(A2T-1-Aj^x)e +e 


= 0 


if and only if 


YT + 1 = e^ 


(3.24) 


The only real solution to (3.24) is Y " 0. Hence det W 5 ^ 0 if and only 
if q_ ^ 0, q„ ^ 0, and A_ A_. The same arguments show that if 



det W ^ 0 if and only if 0, <^2^ ^ ^ ^2* completes 


the consideration of Case II. 


Case III. In this case 


\i ^ At 12 At 

e Te ^ e 


det W = det (qj^T+q 2 )e^^ 


(qj^ ■^+ 

_2 


^TT ^ 

^l^e (q^+Aq^T+Aq2)e ^ (qj^T+q2+Aqj^ ~ +Aq2T+q2^)® 


„ „ 3 T 2At 
-°-l 


Thus det W 0 if and only if ^ 0* 


If \<re. now combine all three cases , it is clear that the eigenvalues 
\ are required to satisfy conditions (i) and (ii) of Theorem 3,5* 
Furthermore, if a pair (q'. A) is observable, then (q'T~^, TAT~^) is 
also observable for any nonsingular T. Take T such that 


TAT = J 


It is kno^m that for observability of (q'T TAT~^) , the components of 
q'T must satisfy precisely the same conditions imposed on the q.’s in 
the above discussion 146]. Hence observability of (q*. A) is also re- 
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quired for the solution of (3.14) - (3.16). Since conditions (i), 

(ii) and (iii) of Theorem 3.5 are necessaxy and sufficient for the 
solvability of (3.14) ~ (3.16) » they are also necessary and sufficient 
for the existence of the matrix B. The proof is finished. 

As an easy corollary of Theorem 3.5, we have 

Corollary 3.5. Suppose n > 3. Given an nxn matrix A, an n~vector q, 
and a nunfljer x > 0, if at least three of the eigenvalues satisfy con- 
ditions (i) and (ii) of Theorem 3.5, and if (q’. A) is observable, then 
there exists a B such that (A,B,q,x) is p.d. on [2x, “)- 

Proof ; Without loss of generality, we may assume that A is in Jordan 
form and in fact the first three eigenvalues satisfy conditions (i) 
and (ii) of Theorem 3.5. By observability cf (q*. A), the first three 
components of q satisfy the conditions imposed in the proof of Theorem 
3.5. If we now consider a solution r to (3.14) - (3.16) in the form of 

(3.25) 


a little thought shows that the equations to he satisfied for the 
solvability of r^, r2, and are precisely the same as those in 
Theorem 3.5. By that theorem, we can solve for r^, r^, and r^ with 
the assumptions in this corollary. Thus a solution to (3.14) - (3.16) 
exists in the form (3.25) and the corollary follows. 
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We make several remarks to clarify the contents of Theorem 3.5 
and Corollary 3.5. 

Remark 3.1 . In contrast to Theorem 3.5, Corollary 3.5 gives only a 
set of sufficient conditions for the existence of the desired B matrix. 
The reason for this is that for n > 3, solvability of (3.14) - (3.16) 
constitutes only a set of sufficient conditions, generally not necessary, 
for the existence of the B matrix. To obtain necessary and sufficient 
conditions in the style of Theorem 3.5 for n > 3 seems to be rather 
difficult and remains an open problem. Nevertheless, it is worthwhile 
to make a few comments on how Corollary 3.5 can be extended* For 
example, if A is 4x4 and has two distinct pairs of complex eigenvalues, 

^1 ” ^2*’ ^3 ^4*’ then condition (ii) of Theorem 3*5 cannot be 

satisfied since there are no real eigenvalues. However, in this case, 
the modified condition 

^(ri^sin^V-^'cos^^T) 0 

where ^ ^ <^an be used instead. We can 

also relax the requirement that (q’. A) be observable. On the other 
hand. Corollary 3.5 is easy to state and apply, and is sufficient for 
our purposes later in Chapter 4. We shall therefore be contented with 
giving Corollary 3.5 as it stands. 

Remark 3.2 . Theorem 3.5 and Corollary 3.5 can be given the following 
interpretation: 

Given A, q, and T, and n ^ 3, there **almost always^* exists a B 
such that (A,B,q,T) is p-d. on [2 t, “) . This is because observability 
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of (q'j A) is a generic property of a pair (q’. A) [47] (conditions 
(i) and (li) are also generic properties of square matrices). ThuSj 
while pointwise degeneracy of the delay system (3.1) is a singular 
property, the existence of a B for which (A,B,q,r) is p.d. on [2x, “) 
is a generic property of A, q, and T, 

Remark 3.3 . The singular cases where no B exists are basically 
related to the eigenvalues of A. For example, if in the 3x3 case, 

A is symmetric and has two equal eigenvalues, then no B exists 
regardless of what q and t are. Furthermore, if B is obtained as 
in Corollary 3.4, it will depend on the eigenvalues of A. This 
remark will be very useful when we apply Theorem 3.5 and Corollary 
3.5 to construct delay feedback controllers in the next chapter. 

Remark 3.4 . If the matrix B is obtained in the manner described in 
Corollary 3.4, the eigenvalues associated with the delay equation 
(3.1), i.e., those values of X such that 

det (Xl-A-e"^’'^B) = 0 (3.26) 

are precisely the same as the eigenvalues of A. To see this, first 

2 

note that (3.14) - (3.16) imply Z = ZAZ = 0. We obtain successively 

det ai-A-e"^'^B) = det[AI-A-e"^'^(AZ-ZA) ] 

= det [Xl-A-e“^’'’AZ+e“^"’^ZA+e"^^^ZAZ) 

= det [Xl+(e"^'^Z-I)A (e"^'^Z+I) ] 

= det(I-e“^”'^Z)(Xl-A) (I+e"^'’^Z) 

= det(Xl-A) 
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By a result of Henry [48], the spectrum of the infinitesimal generator 
of the semigroup for the delay equation is finite, and the range of 
the semigroup is finite dimensional. Kappel [20] has proved a number of 
interesting results for this class of p.d. systems. 

Remark 3.5 . The peculiar condition (ii) in Theorem 3.5 relates the 
eigenvalues of A to the delay time T. ' Indeed, if n = 0, C = ~ » 

^ + e^'' (risin^T-^cos^T) = 0. This suggests that for degeneracy to 
occur, the period of the natural frequency of A should not "match up" 
with the delay time. A more precise interpretation of (ii) is not 
known. 

Theorem 3.5 and Corollary 3.5 give relatively simple and trans- 
parent conditions for the existence of the B matrix. Only simple 
calculations are necessary to check the conditions. More important, 
they give us a hold on when singular situations will arise. As such, 
they form the basis for the development of delay feedback controllers 
investigated in the next chapter. 
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CHAPTER 4 


DELAY FEEDBACK CONTROL OF LINEAR SYSTEMS 
4.1 Introduction 

In the previous chapter, we developed a theorem which allows us, 
under certain conditions, to assert the existence of a matrix B such 
that (A, B,q,T) is pointwise degenerate. This was motivated by the 
control problem described at the beginning of section 3.3. In this 
chapter, we would like to generalize the formulation of the control 
problem to include an input matrix C. Specifically, we would like 
to solve the problem of minimum time output zeroing by delay feed- 
back for the linear system 

x(t) = Ax(t) + Cu(t) (4.1) 

y(t) = q'x(t) (4.2) 

Let us consider the 3 dimensional case. If we just use delay feedback 
of the form 

u(t) = Lx(t-T) 

Theorem 3.5 will impose certain conditions on A, q and T for the 
existence of a suitable matrix L. Furthermore, if we construct a 
matrix B for which (A,B,q,T) is p.d.,.we must also have the range 
of B Included in the range of C in order for L to exist. These 
are rather severe restrictions. On the other hand, since the 
eigenvalues of A play such an important role (see remark 3,3), we 
may be able to solve the problem of minimum time output zeroing if 
we can modify the eigenvalues of A. This suggests using a feedback 
law of the form 
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u(t) == Kx(t) -h Lx(t-T) 


(4.3) 


We shall see that this choice of the feedback law enables us to solve 
the miniinuni time output zeroing problem for (4.1) and (4.2) under 
rather mild conditions. We first study the 3 dimensional cases as 
the required notation is relatively simple, and the techniques used 
can be readily extended to higher dimensions. In section 4.2, we 
establish the form of the matrix B for which (A,B,q,T) is p.d. in 
terms of the parameters A, q and T. We then prove a theorem in 
section 4.3 which gives necessary and sufficient conditions for the 
solvability of the minimum time output zeroing problem in 3 dimensions. 
In fact, the proof of the theorem gives a constructive procedure for 
finding the desired matrices K and L. For greater clarity and com- 
pleteness, a summary of the algorithm is given after the proof of 
the theorem. Next, the techniques are adapted to obtain a solution 
to the control problem in higher dimensions. Again, we summarize 
the algorithm for this case at the end of section 4.4. In section 
4*5, we study the properties of the feedback control system under 
perturbation of its parameters. Finally, we give some examples which 
illustrate the theory. 

4.2 Construction of the Matrix B in 3 Dimensional 

PointwlsB Degenerate Systems 

In the development of delay feedback control using the p.d. 
property, it is very useful to know the explicit form of B. We 
shall carry out the necessary calculations in this section. Using 
these results, we show that in Popovas example (see section 3.2), 
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degeneracy in fact occurs for a unique T and a unique q (up to 
scalar multiples). 


Notation . Since an our p.d. systems will be constructed using 
Corollary 3.4, they will have a degeneracy interval of [2 t,“>). 
From now on, the term p.d. will mean p.d. on the interval [2T,“). 


As in section 3.3, we consider three cases: 
(I) A has three real distinct eigenvalues 
(II) A has two repeated eigenvalues 
and has a Jordan form 


J = 


1 0 


0 

0 0 


Xi 0 


and X2 are distinct and necessarily real. 


(Ill) 


A has three repeated real eigenvalues and has a Jordan form 



1 0 
X 1 
0 X 


Throughout this section, we assume (q’,A) to be observable. By 
Theorem 3.5, we know that there exists- a B such that (A,B,q,t) is 

p.d. 


Case It Let T be the matrix which diagonalizes A, i.e., 


T ^AT = 



0 

0 


0 


0 
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with X^’s real and distinct. Equations (3.14) to (3.16) can be 
rewritten as 

q'TX"^r = 1 
At —I 

q’Te T r = 0 
q’Te'^'^AT“^r = 0 

Letting ot* = q'T, r| ~ T r, we obtain 
a’n = 1 

I At rt 

a'e n = 0 
a’e^*^Ari = 0 

Letting ^ “ 1 , 2 , 3 ^ we can write (4.8) - (4.10) as 


8i + 83 + 33 = 1 

X.T X T X T 

e^e + 32 ® ^ + 336 = 0 


X,T X T X„T 

^1^1® ^2^2® ^3^3® “ ® 



where D = (X--X. ) (e 



Note that D is always nonzero since the real and distinct 

(see section 3.3). 


Next, we construct a matrix Z. 


Z = na'e 


1 

D 


At 




(A3 l+^)t 


a 2A-T 

(. .+X -XjT 




(Xj-Ajja 


(Xj^+X2)t 




(X3 


“1 


Or^2>c. 


(Xj^+2X2“X3)t 


(A^-X2>e 


(Ai*fX2)T 


And B = AZ - ZA 


1 

D 


2X^t 

^(X2-X^)(X3-X^)e 


a 


a, 2X T a (A.,+Xo)T 

^CX,-X2)CA2-X3)e (X1-X3) (X2-X3)e 


«q (X^+XJT 


., (2X +X -X )T a (X +2 X„-X_)t 

(4.19) 


Now Z is related to the original system parameters by 


Z = na'e^^ = T~^rq'TT~^e^'’^T 
= T~^ZT 

and B = AZ - ZA 

= t"^att“^zt-t“^ztt“\t 
= t“^bt 
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(4.20) 


.. i-- 


Thus the desired matrix B is given by 


B = TBT"^ 


We note that the range space of B is spanned by the vectors 


0 

1 

and 

1 

0 H 

®2 


“1 

" "ST ® 

L 3 J 


L “3 J 


(4.21) 


Case II; Let T be the matrix which brings A to its Jordan form, i.e,, 


T ^AT = J = 


1 0 
0 0 


L 0 0 x^J 

Completely analogous arguments show that the desired matrix B is given by 


B = T 


rigWie 


X^T 


[Ti2(a3^T+a2)“n3^a^]e 




i-(Xl 


x^tI 








X2T 


„-l 


XiT 


[ (X2-X^)ri3(a^T+a2^"’^3“l^® 


X^T 


(4.22) 


where a* = q’T and the rj.’s satisfy 




A. 

E(X2-X^)T +g-(X2-Xj^)-l]e 2 

O^D ^ 


(4.23) 
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( 4.2 




^3 a^D 

X„T X-T 

with D = ^ 


The range space of B is spanned by the vectors . 





Case III: Let T be the transformation matrix such that 


X 1 0 


T AT = J = 0 X 1 


0 0 X J 


I Ti2a3^e [ t ]^ (a^T+a2 ) -n^ot^ ]e [TI2 (otj^ — “^1 ^ ® 


TT 

B = T n^otj^e [n 3 (a^T+a 2 )-Tl 2 «ile [ 113 ( 01 ]^ ^ -H3t2T+a3)-ri2Ca3^T+a2)]e 


0 ~Ti 30 C^e 


-n3(a^T-Hx2)e 


where a’ = q*T and the r|.’s satisfy 



( 4.31 


^2 = 


-2(a^T + 


2 2 
ttiT 



The range space of B is spanned by the vectors 


( 4 . 3 ; 


’’^ 2 ' 


["il 

^3 

and 

to 

0 


J^3_ 


It is important to note that the transformation matrix T will in 
general depend on A. Thus the expressions for B given above do not 
show B as an explicit function of A* However j the form that B must 
assume will be useful in our subsequent analysis. 

As an application of the above constructions let us re-examine 
Popovas example* Using the uniqueness of Bj straightf onward cal- 
culations show that for degeneracy to occurs must be 1, and the q 
vector must be such that q^ - We can therefore 


conclude that Popov’s example will be p.d, for a unique vector q, 
up to scalar multiples j and a unique T* 




4.3 Delay Feedback Control of 3 Dimensional 

Linear Systems 

In this section, we apply the results developed in sections 3.3 
and 4.2 to obtain the solution of the minimum time output zeroing 
problem using the feedback law (4.3) for the system defined by (4.1) 
and (4.2). We shall assume throughout this section that the system 
(4.1) is 3 dimensional. As in every theorem related to pointwise 
degeneracy, the result is simple and elegant in three dimensions. 

Theorem 4.1 For matrices K and L to exist such that the closed- 
loop system 

x(t) « (A + CK)x(t) + CLx(t-T) (4.33) 

satisfies q’x(t) = 0, t 2x, for all initial conditions, it is 
necessary and sufficient that rank C ^ 2, and (A,C) be controllable. 

It is helpful to discuss the conditions of Theorem 4.1 before 
the proof. This will help to motivate the construction given below. 
Rank C 2 is clearly necessary since otherwise rank CL ^ 1 and by 
Corollary 3.1, (4.33) is always p.c. Controllability of (A,G) is 
introduced so that we can choose K to shift the eigenvalues of A 
arbitrarily (of course, it is necessary for another reason; see the 
necessity proof of Theorem 4.1 below). This enables us to avoid the 
singular situations described in Theorem 3.5. For example, if C is 
3x3 and of full rank, then obviously we can find a matrix K such that 
the conditions of Theorem 3.5 are satisfied. Furthermore, if we con- 
struct a matrix B such that (A+GK,B,q,r) is p.d. , then since C is 
nonsingular, there always exists a matrix L such tnat B = CL. Of 
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more interest is the case where rank C = 2. We can still choose K 


such that (A+CK) has eigenvalues which satisfy conditions (i) and 
(ii) of Theorem 3.5. However, it is not clear a priori that there 
will always exist L such that B = CL, where B is such that (A+CK,B,q,T) 
is p.d. Indeed, most of the work involved in proving Theorem 4.1 is 
to show we can choose the eigenvalues of A+CK in such a way that the 
matrix B, constructed as in section 4.2, always lie in the range space 
of C. 


We shall need the following lemmas whose proofs are given in 
Appendix A. 

Lemma 4.1 Let 0,23 be real numbers with the property 
0.2 - ^ 0* Then the function f(X) = (a^ + a 2 X + cx^X )e'^ for 

A real has one maximum and one minimum. Furthermore there exist 
straight lines of the form g(A) = S(A) “ each of which 

intersect f (A) at three real distinct points, none of which is a zero 
of f(A). Here 3 is a finite real number not equal to any of the zeros 
of f(A), Yg is some real number dependent on 3> and w is some real 
constant. 

Lemma 4.2 Suppose we are given a 3-vector q, a 3x3 matrix A, and 

a 3x2 matrix C of full rank, with (A,C) controllable. Suppose q'c^ /= 0 
til 

where is the i column of C* Then there exist matrices K and P 
-1 

such that P (AfCK)P is in companion form^ q’P = ctg cx^) is such that 

2 "1 th 

0 L 2 - 4a^tt2 > 0, and that P ‘"'C has as its i column 


‘ 0 " 

0 



Furthermore, if the other column can be written as 



^2 

with Os then P can be chosen such that is not a zero of 

nU) “ cx^H-a2^ ^3^^* 


Proof of Theorem 4*1 (Necessity) That rank G ^ 2 is necessary has 
already been discussed above. If (A,C) is not controllable, (A+CK,C) 
is also not controllable for any K since constant state feedback has 
no effect on controllability. Noting that 

(CL ACL..., a’^'^CL) = (C AC. a“"^C)L 
we see that (A-K3K,CL) is also not controllable for any L. Since by 
Theorem 3.4, all 3 dimensional p.d. systems are regular, this implies 
that the feedback system (4.33) is p.c. for any K and L. Thus con- 
trollability of (A,C) is necessary. 


(Sufficiency) By the discussion following the statement of Theorem 
4.1, it suffices to consider the case where C is a 3x2 matrix of full 
rank. We give a constructive procedure for finding the matrices K and L. 

Step 1 ; We may assume, without loss of generality, that q’c^^ 0. We 

choose matrices and such that P^^(A+CKj^)P^ is in companion form, 
Pj^^C is of the form 

-0 Sj, 

0 82 

_1 B3 
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aad = (*^2 ^2 ^3^ with - 4a20t^ > 0. By Lemma 4.2, such 

matrices exist (see the proof of Lemma 4.2 in Appendix A for the 


-6 


(4.35) 


2 

+ OgX + = 0 

2 

This implies q'P^p 2 “ ^^2 where = ctj^ + + “3^i* 

components all nonzero. By Theorem 3.5, we can construct a matrix 
B such that (A,B,q'P^P 2 ,T) is p.d. In fact, the results in section 
4.2 show that B is given by 


1 

D 


Hi 2X.T 


n« 2 X„t ri, CA„+X,)t 

(Xj-Xj). 


n, (X,+x„)t 

^(X2-X3)(X3-Xpe 


n^CX^ Xi)(Xj^-X2)e 


(2X,+X„-X-)t rio CX,+2X5 “X„)t 


X,T X„T (X„“X„)t X^t X-t 

where D = (X^-X^) (e -e ^ ) + (X^-X 2 )e (e -e ^ ) 


Step 5 : We now show that there exists a 2x3 matrix such that 

-1 -i 

P 2 P^ CLj^ = B. We know from section 4.2 that ^(B) , the range space 
of B, is spanned by the vectors 


0 


1 

1 

and 

0 

"2 



_■ "3 ' 


"l3 


A necessary and sufficient condition for the existence of is that 
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^(B) C^(P 2 C). Since dim^%(B) = dim^(P 2 P^^ C) = 2, 

exists if^(B) =0ti^^ P^^ C) . Thus we only need to find a 2x2 


nonsingular matrix 




^11 ^12 


^21 ^22 


such that 



^11 ^12 
p p 

^21 *22 J 


0 

1 

ri 2 C^ 2 "^ 3 ^''^ 



1 

0 



The first colum of is then given by 


T n, 2 X T 


“11 


“21 


given by 


1 ^^2 ^^ 2 ^ 


12 


L ^22 


given by 


. The second column of is 


. The last coltimn of is 


n n )e 


'11 


1 ^3 


^ c\.-xja,-xje 


r 


CXi+X3>t 


(X 2 +^ 3 )t 



-1 


Using the particular forms of and ?2 we obtain 



^1^22 

^2^21 

^2*22 

^ll'^V21 

^12'^^3^: 


TI 2 

1 - — e 
^3 


^2 ^^2 ^ 3 ^*’- 

V^3 TI 3 ® 


’^1 

1 - — e 


X,-X„ e 
1 3 ri3 


2 ,2 ^2 ^2 ,2 


^2“^3 ® 


We consider three cases 


Case A. = 0. 


This requires 


"^2 ^ 
l-_e =0 


n, (X,-X-)T 

and 1 - — e =0 

'’3 


Xf-X" -- e 
1 3 Ti 3 


C4.36) 


These can be rewritten as 


^9*^ ^ 3 "^ 

“ ^ 2 ® ” ’^s® 


(4.37) 


Since - 40^03 > 0, by Lemma 4.1, we conclude that there exists a 

2 X*T 

constant w such that the line X => w intersects (cx^+a2X + .a3X )e 
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at three real distinct points X 2 , and X^- 
eigenvalues that we choose. We can solve for 


from (4.36) explicitly by 


= 


^2"^3 


21 e. 


X 1 -X 3 

0 - ^ i 

^22 6, 


^11 ~ ^2 “ ^3 ^ 3^21 


^12 " ^1 " ^3 “ ^ 3*22 


These will be the 
\l’ ^ 12 ’ ^21’ ^22 


Case B . &2 “ 

This requires 

XT XjT X3T 

TlnS 11,6 n,e 

•^Sr- ° r ~ =-^~ ir ~ 

Ai A2 Ag 

By appealing to Lemma 4.1, x^e conclude that real and distinct X^, X 2 
and X^ satisfying (4.38) exist. With the eigenvalues thus chosen, the 
matrix L 2 can be obtained from (4.36). 

Case C. nonzero. 


Completely analogous considerations shoxi; that xre must have 



We have now constructed a p.d. system of the form 


z(t) = Az(t) + CL^z(t-:) (4,40) 

with q'P^P^zCt) =0 t > 2 t 

In terms of the original coordinate system, we have 

x(t) = [AH-C(K^+K 2 ?^^) ]x(t) -t CLj^P^^^^xCt-T) (4*41) 

“1 

is p*d. for q* The desired feedback matrices are thus K = 
and L = theorem is proved* 

For greater clarity and completeness, we summarise the algorithm 
for constructing the matrices K and L in the three dimensional case* 

We assume, without loss of generality, that ^ 0* 

(i) Construct, as in the proof of Lemma 4.2 in Appendix A, 

-1 -1 
such that P^ (A + is in companion form, P^ C is of the form 



2 

and q'P-|^ = (ct^ ot 2 “ 3 ^ with a,^ - 4aj^a^ > 0 . 

2 

(ii) Let nCX) = . If 3^ = 0, choose three real 

and distinct numbers ^ 2 * and such that n(X^) 0 , i=l,2,3, 

XlT X„T XqX 

and that ri(X^)e = n(X 2 )e * = r)(X 2 )e . If ^2 “ choose real 

and distinct X^^, X 23 and X^ such that n(X^) Oj i=l, 2 , 3 » and 
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If 6^ and &2 both nonzero, choose real and distinct X2, A^ 
such that n(X^) 7^ 0 , i=l, 2 , 3 , and 

/nCX^) "\^3 

(iii) Once X^, X^, X^ are chosen, choose a matrix such that 
the matrix P“^(A + eigenvalues Xj^, X2, an 

(iv) Compute the Vandemonde matrix 



(v) Calculate the matrix as in step 5 of the proof. In 
other words, find such that (P2^ + CK^ + 

P~^ ^1^ ^^1> ^2 'I’ choice of X^, X2, and X^ in 

(ii) guarantees that exists. 

(vi) Finally, the desired feedback matrices are given by 
K = Ki + KjPf 
and L = L^^P^^ P]^^ 

Note that once the eigenvalues X^, X2, and X^ are chosen, the cal- 
culations involved in steps (iii) to (vi) are completely explicit 






and straightforward. The calculations involved in step (i) are 

detailed in the proof of Lemma 4.2 in Appendix A. They are all 

explicit excepting perhaps the choice of fj^ and f 2 < The numbers f^ 

2 *^22 

and f 2 are chosen to satisfy ~ ^ 0 ^ being a 

2 

zero of + a 2 X + a^X . A simple search procedure will enable us 

to find an appropriate f^ and f 2 . As for step (ii) , we need to find 
eigenvalues X^^, X 2 » X^ which satisfy (4.37), or (4.38), or (4.39). 

One way of doing this is to plot, as in the proof of Lemma 4.1 in 
Appendix A, the function f(X) and draw the appropriate straight 
line to intersect f(X) at three real distinct points. While other 
methods for solving (4.37) - (4.39) can probably be devised, the 
computations involved In step (ii) will, unfortunately, be nonexplicit. 

Remark 4.1 The proof of Theorem 4.1 is rather clumsy and involved. 
Since we only assume controllability of (A,C) we only have freedom 
in assigning the eigenvalues. The lengthy construction shows that 
we can find a suitable set of eigenvalues so that the matrix B we 
constructed necessarily lies in the range space of the input matrix 
C. Certainly a simpler and more elegant proof is desirable. However, 
because of the nonlinear nature of the problem, a substantial simpli- 
fication may not be possible. 

Remark 4.2 Remark 3.4 shows that the eigenvalues of the delay feedback 

_3_ 

system (4 •41) are precisely those of [A + ^(K^ + ^ 2 ?^ )], and these 
satisfy (4.44) or (4.45) or (4.46). In general, there are many 
solutions to these equations (see Appendix A) . If we can choose the 
eigenvalues so that they are negative, we will have obtained an 
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asymptotically stable p.d. system. We have not been able to show as 
yet that such a choice is always possible. Thus, the closed-loop 
p.d. system (4.41) may be unstable. 

Remark 4.3 We have assumed that the delay time T is prescribed in 
advance and not available as a design parameter. However, if it is 
to be designed, the smaller T is, the larger the elements of the B 
matrix will be (see eqs. (4.19) and (4.17)). Hence we have a tradeoff 
in this situation. We shall not investigate this point further. 

4.4 Delay Feedback Control of Higher Dimensional 
Linear Systems 

Our goal in this section is to extend the feedback control results 
for 3 dimensional systems to higher dimensions n > 3. The method of 
proof is substantially the same as that of Theorem 4.1. Notationally 
however, it is much more involved. 

Theorem 4.2 For matrices K and L to exist such that the closed-loop 
system 

x(t) = (A + CK)x(t) + CLx(t-T) 

satisfies q’x(t) = 0, t ^ 2 t, for all initial conditions, it is 
sufficient that rank C ^ 3, and (A,C) be controllable. 

Proof: Consider the equations 


q'r = 1 

(4.42) 

, At =0 
q e r 

(4.43) 

II 

o 

(4.44) 
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where q* = (q^ q^... q^^) 


A = 


0 


0 0 


• * 


X , 0 

n-1 


n 


Suppose at least three of the eigenvalues X^ are real and distinct 
with the corresponding q^*s nonzero. Without loss of generality, 
we may take X^, X^, and X^ to be these eigenvalues with q^, qgj q^ 
nonzero. Under these conditions, (4.42) - (4.44) can be solved to 
give 


= 


3 q3W3 


(A.+X„)t 

(X 2 ~X^)e ^ ^ n 


XnT X,T "X) 


, '‘1 . 
(e -e ) 


i=4 


(4.45) 


^2 “ X„T X-T 

q 2 (e -e •^ ) 


X-T n X.T X^T 

i=3 


(4.46) 


r, = 1 - 


Z Vi 


i-2 


where 


w, 

X 


X-T X-T X-T X,T X-T X T 

CXj^-X^)e ^ (e ^ -e ^ )-Ce -e ^ ^ 

V V 

e - e 


(4.47) 


(4,48) 
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Thtis T 2 and are completely determined in terms of the 2n-3 
’’free” variables ^5>*’**^n* Constimcting 

Z = rq*e^^ and B = AZ - ZA we obtain 


’^i^V^2^v • * 




XlT 


B ^ 


^3 ^2^ 


X2T 


X T 

r T (X _-X )a e ^ 
n-1 n-1 n"^^ 


r (X -X. )q,e 
n n 1 ^1 






V 


Note that B is of rank 2, and its range space is spanned by the vectors 





0 


ri(X^-X2) 



0 

* 

» 

• 

* 

and 

raCXs-Xz) 

* 

• 

r (X -X.) 
n' n r 


* 

_ __ 





We now proceed as in the proof of Theorem 4.1. We need only to 
consider the case where C is nx3 of full rank. Construct matrices K 
and such that (A + BK)Pj^ is in companion form T^th distinct 


eigenvalues , and 
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such that ( ,B,q'Pj^P2, ) is p.d. Analogous to step 5 of the proof of 


Theorem 4.1, we see that we need to find a matrix such that 


^1 ^2 " ^2 


^2 ^^2 ^ 1 ^ 


r (X -XJ 
n n 1^ 


0 






E 

i^l 

E 


r . (X.-X- ) 
x' 1 1 ' 


riX^(X.-X^) 


y^. r±X^ 

i^^l 


E 

W2 




E 

±^2 


T.x^ ^(X.-X.) 

XX X 2'' 


Let 


^2 = 


'11 

^12 

'21 

^22 

'31 

^32 
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• . 'i 


Then 


P-^CLJ - 


‘^22^2l'*''^23^31 


d - _J2.„-+d . 

H“lj2 21 n-1,3 31 

^H'*'^n2^2l’’'^n3^31 


*^ 12 ^ 22 ’‘'*^ 13^32 

^ 22 ^ 22 *^ 23^32 


^n -13 2^22*^'^ 
\2^^n2^22^n3^32 


We 


can solve for and 5-^2 independently. Thus we only need to show 


that we can solve for ^22’ ^31* ^32 from the 

equations 

^j2^21^^j3Sl ” S ’^i^i j=l,...,n-l (4.50) 

iT^l 


*^i2^22‘’‘'^13’'’32 S ’^i^i j-l,...,n-l (4.51) 


j2 


j 


i?^2 


We now eliminate ^ 22 * ^31* ^32 (4.50) and (4.51). Using the 

fact that rank C = 3, a little thought shores that we will obtain 2n-6 
equations relating X^, r^,...jr^. Since the ntimber of ’’free" 

variables is 2n-3, we expect that it would be possible to choose the 
X^’s and in such a way that all the requirements are fulfilled. 

The simplest way to see this is to examine the special case where 
d ^2 " 0 3 except j = ra, and d^^ ” 0 i except j = k. 

This gives rise to equations of the form 


i^l j?^,k 


(4.52) 
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) 


and 


i^2 


(4.53) 


If we take X^, and X^ as fixed real distinct constants, these give 
rise to 2n-6 equations in the 2n-6 variables X^,..., X^, r^. 

Eliminate the variables in the above equations. This yields 

n-3 equations in the variables X^,..., X^. Next we note that these 
equations are invariant under permutation of the indices in X^. Hence 
if we reduce these n-3 equations into a single equation involving X^ 
for some i, say of the form f(X^) = 0, then ^CX^) ~ j=4,..*n also. 

However, the form of equations (4*52) and (4,53) implies that f is an 
exponential polynomial [1] and it is known [1] that the zeros of 
these functions are distributed symmetrically about the real axis with 
asymptotic distribution separated at a certain distance. Since there 
is an infinite number of zeros of these exponential polynomials, it 
is possible to choose X^,,.., X^ such that they form a symmetric set 
with the properties . .-f a^X^ ^) f 0, distinct i=l,...,n, 

and X^, X^, X^ real. 


With such a choice, the conditions for the existence of the matrix 

are satisfied and we can construct, from a matrix such that 
“1 “1 

^2 ^2, ^^1 “ While x-76 have only discussed the case where d ^2 “ 

j 1,..., n-1, j ^ m and d. « = 0, j - 1, . . . , n-1, j k, the other 

1 ^ 

cases can be treated in exactly the same way. We can eliminate 


variables from (4.50) and (4*51) in a similar fashion to that dis- 


cussed above and show that the desired X^, X^^***^ exist. The 
details are therefore omitted. The rest of the proof now proceeds 
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in exactly the same way as that of Theorem 4.1. 


We summarize here the algorithm for constructing the matrices 
K and L in the higher dimensional case. Again, we assume q'c^ 4 0. 


,-l 


(i) Choose IC^ such that P^^ (A + CK^)P^ is in companion form. 


"1 

P^ C is of the form 


0 

0 

* 

■ 

0 

1 


12 


22 


n2 


^13 


23 


n3 


, and q'P^ = (o^ “ 2 *** 


(il) Choose eigenvalues such that (4.50) - (4.51) 

a solved consistently for il.., i ; 
is possible by the proof of Theorem 4.2. 


can be solved consistently for il.., i 1,2,3, j = 1,2. Such a choice 

13 


(iii) Choose such that the matrix P^^(A + CKj^)P^^ + 
has as its eigenvalues X 2 ,..., X^. 

(iv) Compute the Vandemonde matrix 


"2 = 


1 

A. 


1 

K 


,n-l ,n-l 
^1 ^2 


^n-1 

n 


(v) Calculate the matrix from 2^^, i = 1,2,3, j = 1,2 such that 
(^2^ P]^^(A+ CK^ + CK2P"^)P3_P2, P-<i- 
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(vi) The matrices K and L are given by 


K = + K2P“^ 




Except for step (ii) , all the calculations involved in the 
algorithm are explicit. To choose the appropriate eigenvalues 
^1’ ^2’**** \ step (ii) ) we can first eliminate ^2l» ^ 22 * ^31’ 

^32’ ^1’ ^2’*"*’ ^n (4.50) - (4.51). This elimination involves 

only algebraic operations. We are then left with n-3 coupled trans- 
cendental equations involving exponential polynomials in the variables 
X^,..., X^. For fixedj real and distinct X^^, X 2 > X^, an iterative 
procedure is now needed to find the appropriate values for X^, X^,..., X^. 
We have not had time to investigate the numerical aspects involved in 
solving these transcendental equations. However, quite a lot of work 
has been done on the solutions of transcendental equations involving 
exponential polynomials. The reader may, for example, consult [1] and 
the references therein. 


Remark 4.4 Theorem 4.2 gives only a sufficient condition for the 
solvability of the feedback problem. It is not possible to obtain 
necessary and sufficient conditions by our approach, since for n > 3, 
solvability of (4.42) - (4.44) is not a necessary condition for p.d. 

Remark 4.5 It may be possible to prove Theorem 4.2 under the assumption 
that rank C ^ 2. A similar argument will then show that we have 2n-4 
equations in 2n-3 variables. However, the algebraic difficulties in 
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shox^ing that these equations can be consistently solved for suitable 
eigenvalues are much more formidable. 

4.5 Sensitivity of Pointwise Degenerate Systems 

In the last two sections, we have ^iven conditions under which a 

delay feedback system xrf.ll have its output identically zero after 2T 

units of time. If these results are to be used in practice, we must 

ensure that the feedback system will behave reasc ibly well under 

perturbation of its parameters. Since we usually ..:iaw which particular 

combination of state variables serves as the output, it is reasonable 

to assume that the output vector q is fixed, and we shall do so. There 

are two questions of interest. Suppose for a fixed set of parameters 

A , C , T , there exist matrices K and L which give rise to a p.d. 
o o o* o o ® 

system (A^ + q, • This yields a (nonunique) "design” 

fxmction f: (A, C, x) i— >(K,L), and a "performance' function g(t): 

(A,C,T,f(A^, C^, T^)) »— }q'x(t) for t ^ 2x. C.nti; ity of f at 
(A^, G^, T^) implies that the delay feedback desii Procedure is well- 
posed. Similarly, continuity of g(t) at (A^, C^, x^, K^, L^) implies 
that small variations in the parameter values give rise to a small 
degradation in the system performance. Note that the continuity 
properties of the functions f and g(t) are related to those of f^ 
and gj^(t), x^here (A,x) B with (A, B, q, x) p.d., and g^(t): 

(A, X, fj^(A^, ) <-)q'x(t), for t ^ 2X. This is because in our 

construction of the matrices K and L, we first construct a matrix B^ 
such that (A + GK, q, x) , is p.d., and then we construct a matrix L such 
that Bj^ = CL. If C is the identity matrix and K can be taken to be 




zero, then the properties of the functions f and g(t) reduce to those 
of and g^(t). We shall only study the behavior of f^ and gj^(t). 
This section contains some simple results in this direction. 

For the study of f^, we will assume that A has distinct eigen- 
values. This is the case of most interest to us since in our proof 
of Theorems 4.1 and 4.2, we assign distinct eigenvalues to the matrix 
A + CK. We shall concern ourselves only X'jith perturbations of the 
form A + eAj^ = A(e) and x + ex^ =x(e). The following perturbation 
result for matrices is loiown [49]. 

Lemma 4.3 If X is a simple eigenvalue of A, then for sufficiently 
small |e|, there is an eigenvalue X(e) of A(e) with a power series 
expansion 

X(e) = X + + ... 

and there is a corresponding eigenvector x(e) with a power series 
expansion 

x(e) = X + ex^^V. . . 

x^here x is the eigenvector of A corresponding to X. 

A procedure for computing the X^^^*s and x^^^'s from 
may be found in [49] • We can now state 

Theorem 4.3 Suppose A has n distinct eigenvalues X^,,.., 
matrix B(e) = f^[A(e), x(e)] has a power series expansion 
sufficiently small |e| 


A and A^^ 


X . The 
n 

in e for 
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B(e) = B + + ... 

where B = f^(A,x) 

Proof ; Since A has distinct eigenvalues, a transformation T that 
diagonalizes A consists of the n eigenvectors of A. By Lemma 4.3, 
the eigenvectors of A(e) can be expanded in a power series in e for 
sufficiently small |e|. We can therefore also expand T(e) and T ^(e) 
in a power series. This implies q'T(e) can be expanded in a power 
series. Equations (4.45) - (4.49) show that B(e) can again be ex- 
panded in a power series in E. Finally, since B(e) = T(e)B(e)T ^(e), 
B(e) can also be expanded in a power series in e for sufficiently 
small j E I . 

Next, we consider the behavior of g^(t) again under perturbations 
of the form A(e) = A + eA^, t(e) = t + et^^. 

Theorem 4.4 Suppose the system 

x(t) = Ax(t) + Bx(t-x) (4.54) 

is p.d. with respect to q for t ^ 2x. Then the system 

Xg(t) = A(e)Xg.(t) + Bx^(t-x(e)) (4.55) 

will have, for each t ^ 2x, 

q’Xg.(t) = y^.(t) = EYj^(t) + ••• 

Proof: Let z(t) = P^Xg(t-(k-l)x(e))+ ... + ?j^Xg(t) where P^^,,.., Pj^ 
satisfy Popov's condition (3.3) - (3,8). Then 
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z(t) = Pj^A(E)Xg.(t“(k-l)T(E))H-P^BXg.(t-kT(e)) 

P^A(e)Xg(t) + Pj^Bx^Ct-T(E)) 

= Vz(t) + Pj^Aj^Xg.(t-(k-l)T(e))+ . . . +e 

Thus, 

v’z(t) = v’e'^^^^^zCt-xCe)) 

+ eT ®^{P^A^x^(s-(k-l)T(E)) 

A-t (e) 

+ ... + Pj^Aj^Xg(s) }ds 

= v’e^^^^^z(t-T(e)) + eYg.(t) 

Using equations (3.6) - (3.8) in Chapter 3, 

v*z(t) - v'e^^^^^z(t-r(E)) 

= v'z(t)-v’e^^z(t-T(e))“v' (e^^^^^-e^^)z(t-T(e)) 
= q'Xg(t)“v' (e^^^^^-e^^)z(t-x(e)) 

Combining (4.61) and (4.62), we get 

q’x (t) = v'(e^^^^^-e^^)z(t-x(E)) H- 0Y^(t) 

t- E 

The right hand side of (4.S8) is precisely of the form Ey^(t) + 
proving the theorem. 


(4.56) 


(4.57) 

(4.58) 
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Remark 4.6 Theorem 4.4 does not give any estimate of the magnitudes 


of Y 2 (t)> etc. Furthermore, no claim is made on the uniform 

behavior in t of the output Yg.(t). In fact, if the system (4.55) 
is unstable, Yi(t),Yo(t)> etc., which are really derived from x (t) , 
can become large as t gets large. However, if the orii_*-nal system 
(4,54) is asymptotically stable, then for small enough |e|, the 
perturbed system (4,55) is also asymptotically stable [2], In that 
case, Yj^(t), Y 2 Ct), etc., will go to zero as t gets large. 

To a certain extent, the above remark limits the applicability 
of the delay feedback approach we have explored so far. If the 
parameters of the system are perturbed, the errors in the output may 
grow, and hence the performance may become unacceptable after some 
time. However, if in the proofs of Theorems 4.1 and 4,2, we can 
choose the such that^eA^ ^ 0 , then the resulting feedback p*d, 

system will be asymptotically stable. In that case, the above 
mentioned problem does not arise. Whether we can always find feed- 
back matrices K and L such that the resulting closed-loop system is 
asymptotically stable in addition to being p.d. remains an open 
problem. 

4-6 Some Examples 

We give two examples which illui^trate the use of delay feedback 
in certain problems Ir linear system theory. 

Example 1 : ^Deadbeat” output control- 

Consider the system 
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x(t) = Ax(t) + Cu(t) 


Inhere 



y(t) = q'x(t) 


with q' = (2 -2 1) 

The problem is to drive y(t) to zero in minimum time by delay feedback 
control of the form 

u(t) = Kx(t) + Lx(t-l) 

Let us note that (A,C) is controllable and rank C = 2, Also A 
violates condition (i) of Theorem 4.1. Thus we cannot solve this 
problem by using only delay feedback control (i.e., K = 0). 

One convenient choice for K is 



and a suitable B is 



we see that CL “ B, 

With these choicesj one can easily verify that (A + CK, CL, q, 1) 
is The output y(t) has been controlled to zero and T^ill remain 

zero for all t ^ 2. 

The problem that is in a sense ”dual” to the deadbeat control 
problem is that of a "deadbeat" observer. Consider the system 
x(t) = Ax(t) 

y(t) = Cx(t) 

We would like to reconstruct the state of the system. The usual 
approach is, under the assumption of observability, to construct 
an observer of the form 

x(t) = Ax(t) + K[y(t) - Cx(t)3 

such that the estimation error e(t) = x(t) - x(t) goes to zero 
asymptotically. The theory developed here suggests the use of 
a delay observer of the form 


-9 




x(t) = Ax(t) + K[y(t)-Cx(t) ] + L[y(t-T)-Cx(t-T) ] 

such that a linear conifaination of the error, q*e(t), is driven to 
zero in minimum time. If ue can observe all but one component of 
the error, q’e(t), is driven to zero in minimum time. If we can 
observe all but one component of the error vector, this will enable 
us to reconstruct the estimation error and hence the state of the 
system in finite time. While finding necessary and sufficient con- 

ditions for the existence of such a delay observer is an open problem 
at the present, we can give an example illustrating the technique 
involved. 

Example 2 i "Deadbeat” observer. 

Consider the system 
x(t) = Ax(t) 

y(t) = Cx(t) 

where 

-12 0 

2 -2 -1 

12 0 

10 0 
0 2 0 

(A,G) is observable and rank C = 2. Since x^^ and X 2 are directly 
observed, we need only estimate x^. Using the usual reduced order 
observer, we can asymptotically reconstruct the state by a 1 di- 
mensional observer. 
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Here we shall use delay feedback. Consider an observer of the 


form 


x(t) = AxCt) - K[y(t)-Cx(t) ] - L[y(t-1) - Cx(t-l)] (4.59) 


Then the estimation error e(t) = x(t) - x(t) satisfies 


e(t) = (A + KC)e(t) + LCe(t-l) 


(4.60) 


We want to find K and L such that (4.60) is p.d. One such choice is 


Then 


K = 


L = 


1 

-2 

-1 

0 

1 

0 


A + KC 


LC = 


0 

1 

-1 

0 

0 

1 


0 

0 

0 


0 

1 

0 


0 

0 

2 


2 0 
0 -1 


0 

0 

0 


Equation (4.60) becomes precisely Popov's example, and it satisfies 


e^(t) - 2 e 2 (t) - e 2 (t) = 0 


t > 2 
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But x^Ct) = x^Ct) + e^Ct) 

= x^Ct) + 6j^(t) - 202 (t) for t ^ 2 

= x^Ct) + yj^(t) - Xj^(t) - [y 2 (t) - 2 x 2 (t) I for t ^ 2 

We know Xj^(t), X 2 (t), and x^Ct) from the solution of the observer 
equation (4.59). Hence x^Ct) is reconstructed after 2 units of time. 

The tradeoff betwfeen using the delay feedback observer (4.59) and 
the usual reduced order observer is clear. Delay feedback allows us 
to determine the state exactly after 2 units of time. Hoi^everj the 
delay feedback observer must be 3 dimensional. 
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CHAPTER 5 


OPTIMAL FILTERING OF LINEAR AND NONLINEAR 
STOCHASTIC DELAY SYSTEMS 


5.1 Introduction 

After studying various deterministic aspects of the theory of 
delay systems in the previous chapters, we turn our attention to 
stochastic problems. In this chapter, we study the filtering problem 
for stochastic delay systems of the form 

dx(t) = f(x^,t)dt + H’(t)dw^(t) 

x(0) = X (6) 0e[-T,O] (5.1) 

o 

dz(t) = h(x ,t)dt + N(t)dw_(t) 
t L 

z(s) = W 2 (s) =0 s ^ 0 (5.2) 

We assume the reader to be familiar with the basic elements of stochastic 

processes [50], and we simply collect the basic definitions here for 

future reference. All stochastic processes are defined relative to a 

given probability space (fl,^P) and on an interval of the form [0,T]. 

The system process x(t) takes values in R*^, the observation process 

z(t) in R^. For simplicity, we take w^^(t) and W 2 (t) to be standard 

Wiener processes in R® and R^ respectively, completely independent of 

each other. The initial function x is taken to be some random function 

o 

on [-T,0], completely independent of w^^(t) ^nd W 2 (t). The maps f and h 
are functionals, possibly nonlinear, defined on'8’x[0,Tl. H(t) and N(t) 
are mxn and pxp matrix-valued continuous functions respectively. Further- 
more N(t) is assTimed to be symmetric and positive definite. 
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We shall write dw(t) = H’ (t)dWj^(t) and dv(t) = N(t)dw 2 (t). 
Define Q(t) = H'(t)H(t), R(t) = N^(t). With this notation, w(t) 
is an "unnormalized" Wiener process in with 

rmin(t,s) 

cov[w(t) ; w(s)] = I Q(u)du 

Similarly, v(t) is an "unnormalized" Wiener process in R^, with 

J pmin(t,s) 

R(u)du. This notation will be used 

0 

throughout this chapter. 

The basic filtering problem is to estimate some function (j) of 
x(t) given the obsetnrations z(s), 0 ^ s _< t. It is well-kno\inn. that 
the optimal estimate with respect to a large class of criteria, for 
example minimum mean square error, is the conditional expectation 
E{(j)[x(t) ] [z }, where z denotes the (^-algebra generated by the ob- 
servations z(s), 0 ^ s ^ t. We shall also write E{<|>(x(t)) |z^} as 
$[x(t)] or E^{(j) [x(t) ]} . Our objective in this chapter is to obtain 
formulas for $[x(t)] in the case of delay systems. 

The nonlinear filtering problem has been extensively studied 
by many authors. In particular, Fujisaki, et al. , [51] have given 
a stochastic differential equation for the evolution of <|)[xCt)] 
for rather general stochastic systems, which includes our delay 
model. Specifically they showed that for the delay system (5.1) 
and (5.2) 
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(5.3) 


R ^(t)[dz(t)-h(x^,t)dt] 

where is a differential operator. If there are no delays in the 
system, the unknoxm terms on the right hand side of (5.3) are of the 
form (j)[x(t)]h[x(t) ,t] and, assuming suitable differentiability con- 
ditions, one can write another stochastic differential equation for 
f [x(t) ]h[x(t) ,t]', leading, in general, to a countably infinite-order 
system of moment equations. While there is no known method of analyzing 
this infinite set of equations, at least the way in which they arise is 
clear. One can thus use approximation techniques to study these 
equations, and develop suboptimal filtering schemes. 

Unfortunately, when there are delays in the system, the above 
procedure does not go through. The reason is that one of the terms 
on the right hand side of (5.3) is sud since h is a 
functional on the segment x^, it is not clear how one can develop a 
stochastic differential equation for 4*[x(tHh0c^Tt^ using (5.3). 

These difficulties motivated the development of a representation 
theorem for a functional of x^. Using this representation, we derive 
stochastic differential equations for certain functionals f and h. 

The linear case with Gaussian distributions is then studied in detail. 
Stochastic differential equations generating the conditional mean as 
well as partial differential equations for the conditional covariance 
are derived. This will prepare the way for the filter stability 
discussions in the next chapter. We will also discuss some of the 
basic difficulties in deriving stochastic differential equations for 
general nonlinear functionals ^(x^). 
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->•2 Existence, Uniqueness and Other Properties of Solutions 
of Stochastic Functional Differential Equations 

In order for our estimation problem to be well-defined, we need 

conditions which guarantee existence and uniqueness of solutions to 

the functional stochastic differential equations (5.1) and (5.2). 

Since (5.1) can be solved independently of (5.2), we shall be mainly 

concerned with conditions to be imposed on (5.1). Conditions on (5.2) 

will be stated at the end of this section. 


Existence and uniqueness of solutions to stochastic functional 
differential equations of the type (5.1) have been studied by Ito and 
Nisio [52], and Fleming and Nisio [53]. We state their results here. 
Assume 


(A.l) 
(A. 2) 


£(<f),t) is measurable and continuous on^x[0,T]; 

there exists a bounded measure F on [-T,0] and a positive 

constant K such that 


|f(4>,t)-fC|),t)l 


< K 



)-i|j(s)|dr(s) 


(A. 3) on the interval [-T,0], x(t) is continuous w.p.l with 
e|x(0)|^ < ”, -T £ 6 £ 0. 


Proposition 5.2.1 Assume (A.l) to (A. 3) are satisfied. Then there 
exists a unique solution to (5.1) which is continuous w.p.l and has 
bounded second moment, and 

E|x(t)|^ £ for some positive y < «>, 

Furthermore x^ is a Markov process. 
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The proof of Proposition 5.2.1 can be found in Ito and Nisio [52] > 


or Fleming and Nisio [ 53 ], and Kushner [54]. 

Let us now study some of the properties of linear functional 
stochastic differential equations which are relevant to our later 
investigations. We consider the equation 

dx(t) = a(x^,t)dt + dw(t) (5.4) 

where a(x^,t) is given by the Stieltjes integral 
0 

dgA(t,0)x(t+0) 

-T 

Here, A(t,0) is a function on RxR, jointly measurable in (t,0), 

continuous in t, of bounded variation in 0 for each t, with 

Var A(t,*) <. m(t) , a locally integrable function on where Var 
[-T,0] E~t,0] 

means the total variation in [~T,0]. Furthermore, A(t,0) = 0 for 0^0, 

A(t,0) ='A(t,-T) for 0 “T, and it is continuotis from the left in 0 on 

(-T,0). We shall prove a type of variation of constants formula for 

(5.4). We first prove a lemma which will be used later and may be of 

some independent Interest. 


a(x^,t) 




Lemma 5.2.1 Let G(t,s) be a deterministic function on RxR such that 
2 

I [G(t,s)| ds < for all te[t^,T], G(t,s) is continuously differ- 


'to 

entiable in t, and 


• 2 

[G(t,s)|*dt ds < «>(GCt,s) 

■ 

■ / 


= -^G(t,s)), 


Then the Wiener integral g(t) “ f G(t 5 s)dw(s) has a differential on 
[t^,T] ""c 
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■f 


dg(t) = I G(t ,s)dw(s)dt + G(t,t)dw(t) 

't. 


(5.5) 


Proof : Proving (5.5) is equivalent to proving that for any t^j t 2 in 




6(^2^ “ SCt^) “ 


rr 

1 o 


f 


G(t js)dw(s)dt + I G(t,t)dw(t) (5.6) 
't. 


By the assuntpcion of square integrahility of G(t,s), the Fubini type 
theorem for Wiener and Lebesgue integrals [50], is valid, and so 

/-t , r^i 

G(t,s)dw(s)dt =1 I G(t,s)dt dw(s) 


n 


't^ •'t 
1 o 


r r 


'o ‘-1 


*^2r^2 

+ I I 6(t,s)dt dw(s) 


■r 


G(t2,s)dw(s) 


-f 


fY 

•'t, Js 


G(tj^,s)dw(s) + 


r 


G(t2,s)dw(s) 


£ 


G(s,s)dw(s) 


The right hand side of (5.6) now yields 


r ^ G(t 2 ,s)dw(s) " r ^ G(t^,s)dw(s) 
*'t_ *^t 


which is precisely the left hand side of (5.6), 


Using Lemma 5.2,1, we can easily obtain a variation of constants 
formula for (5.4). Let $(t,s) be the fundamental matrix associated 
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with (5.4), i.e., that unique matrix which is absolutely continuous in 
t, essentially bounded in s, and such that 


9i(t,s) 

3t 


^(t 


/“ 

|o 

,s) = { 

U 


dgA(t,0)#(t+0,S) 


for S“T < t<s 


t > s 


t=s 


A discussion of the properties of $(t,s) may be found in Hale [2]. 


Theorem 5.2.1 The solution x(t) of (5.4) can be written as 


x(t) = #(t,0)x^CO)+ 




. f 


$(t,s)A(s,B-s)ds)x^(0) + I #(t,s)dw(s) 


(5.7) 


Proof : The fundamental matrix $(t,s) satisfies the conditions required 
in Lemma 5.2.1. We may therefore apply Lemma 5.2.1 to the right hand 
side of (5.7) to obtain 


1 “ 

‘'-T 


dx(t) = dgA(t,0)$(t+0,O)x^(O)dt 


£ €£ 


dgA(t,6)i(t+e,s)A(s,B-s)ds>x^(B) 


t rO 


fS 

Jq J-x 


d_A(t,0)$(t+0,s)dw(s)dt + dw(t) 

y 


Applying the unsymmetric Fubini theorem of Cameron and Martin [55 ] , we 
obtain 
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d 0 A(t, 9)x(t+0)dt + dw(t) 



dx(t) “ 



so that (5.7) is indeed a solution of (5.4). 


Remark 5.2.1 Similar variation of constants formula for linear 
stochastic functional differential equations have been obtained by 
Lindquist [56 ] . All are related to the representation of solutions 
of linear functional differential equations given by Banks [57], 

The mean of x(t) , x(t) , can easily be seen to satisfy 
x(t) = r dgA(t,6)x(t+0) 

J-x 

x(0)=x^(0) 0e[“T,O] (5.8) 


whose solution is given by 


x(t) = ®(t, 0 )x^( 0 )+ 


i: ‘-ir 


5>(t,s)A(s5 3-s)ds>x C3) (5.9) 


! 3C I 


As for the covariance associated with the solution of (5.6), we 
shall be interested not only in the covariance of x(t) , i.e., 
E{[x(t)“x(t)] [x(t)-x(t) 3 ’}, but also in the covariance "operator” 
2(t,0,O ** E{[x(t+0)-x(t+e)][x(t+C)-x(t+C)]’}, -T < 9, 5 < 0. 
Heuristically , this corresponds to the covariance of x^. We first 
compute it and then give it a more precise interpretation in terms 
of the characteristic functional of the ^-valued random variable x^. 


Let e(t) = x(t)“x(t) then from (5.7) and ( 5 . 9 ) , and using the 
fact that $(t,s) = 0 , t < s, we get that 
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e(t’^Q) = 


4Ct,0)e(0) + I 


i 


c-c 


$(t+6,s)A(s,p-s)ds|e(3) +f O(t+0,s)dw(s) 

•^0 


Using the independence of and w(t), we get 


2(t,e,C) = E{e(t+0)e*Ct+e)} 


= $(t, 0)2(0, 0,0)^’(t,0)+ 


jC€ 


®(t+9,s)A(s, p-s) da 


I 


•2(0,e,0)$' (t,0) 


f ®(t,0)d^ f 2(0,0,3)A'(s,B-s)$'(t+e,s)ds 

J-x Jo 

I --ii: 9 (t+0 , s) A(s , B-s ) ds| 2 (0 , B j Ot) 






f 

J 0 


+ 1 4»(t+0,s)Q(s)®' (t+5,s)ds 


(5.10) 


If we view x^ as a ^-valued random variable, then its character- 
t 

istic functional <j)(y,x^) is defined by 
<|)(y 5 X^) “ E exp i <y, x^> 

where y is an element of'g’*, the dual space of % which is the space 
of functions of bounded variations on [“T,0] with y(0) = 0, and <,> 


is the pairing between®* and defined by 

fO 

<y,x> = I dy(0)x(e) 

•'-T 
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dy(0)x(t+6) is a Gaussian 


Recognizing that for each t, <y, x > = 


= f 


random variable, we can easily evaluate 




1/2 Q^(y) 


(5.11) 


where is given by (5.9) and Q^(y) is a quadratic form in y given by 


Q^Cy) = 


-ff 


dy(0)ZCt,0,Ody^C^) 


C5.]2) 


The form of <j)(y»x^) implies that we may interpret as a ‘S'-valued 
Gaussian random variable for eact t, with E(ta0j5) as its covariance 
operator. These considerations will be useful later in section 5.5 
when we study the linear filtering problem for delay systems. 

We now state conditions concerning eq. (5.2). 


(A. 4) h(^,t) is measurable and continuous on®k[0,T]. 

-T 


(A. 5) 


I ^ 

-/n 


E[h(x^,t)’h(x^,t)ldt < » 


We note that (A. 1) - (A. 5) are not the weakest assumptions for which 
our results are valid. However, we have not striven for more gener- 
ality because this ^d.11 only introduce technical complications without 
adding insight into the optimal filter structure for delay systems. 
From now on, (A. 1) to (A. 5) will be assumed to hold. 


5.3 A Representation Theorem for Conditional Moment 
Functionals 

In this section, we derive a representation theorem for the 
conditional expectation of functionals of the form ^(x^) given the 
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observations zCs), 0 _< s ^ t. As beforej we shall denote o{z(s), 0 ^ s ^ t} 
by z ^5 and we shall call objects of the form E{^(x^) |2^} conditional 
moment functionals * Our approach makes use of the Girsanov measure 
transformation technique as given in Wong [58]* Using the represent- 
ation theorem so derived, stochastic differential equations can be 
obtained for suitably smooth functionals 0, f, and h. 

We first note that we can reT'jrite (5.2) as 

N"^(t)dzCt) = N"^(t)h(x^,t)dt + dw^Ct) (5.13) 

Clearly z(s), 0 ^ s ^ t and f N~^(a)dz(a) = Zj^(s) , 0 s ^ t 

-^0 

generate the same a-algebra. Define h^(x^,t) = W ^(t)h(x^,t). 

In deriving the representation theorem, \<j& shall use (5.13), i.e., 
z^(t) and h^(x^,t), rather than (5.2). The reason for introducing 
z^(t) and h^(x^,t) is that the absolute continuity results given 
in Lemma 5.3.1 are phrased in terms of standard Wiener processes. 

Define a new measure on (S?,^ by the formula 


dP ^T 

-^ = exp [- I h^(x^,t)dw 2 (t) 
•^0 



h^(x^,t) *h^(x^,t)dt] 
(5.14) 


Lemma 5.3.1 P^ is a probability measure with the following properties: 

(a) Under P^, z^(s) , 0 ^ s ^ T has components that are in- 
dependent standard Wiener processes. 

(b) Under P^, the processes z^(t) 0 £ t _< T aad x(t), 0 t T 
are independent. 

(c) The restriction of P^ to a{x(s), “T ;< s T is the- same as 
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the corresponding restriction of P. 

(d) P « P and 
0 


dP 


esp[ 



hiCx^.t) *dzi(t) 



hj^(x^,t)h^(x^,t)dt] 


(5.15) 


Proof : The conditions imposed on the processes involved give rise to 
a virtually identical situation to that of Proposition 5.1j Chapter 6 
of Wong [58]. The proof there applies without change to our case. 


Let us denote integration with respect to P and by the ex- 

pectation operators E and respectively. The reader should note 
that the true measure relevant to our estimation problem is ?. The measure P^ 
is strictly a device for obtaining the representation theorem. 


Let us write ^ for the least 0-algebra containing ofzCs), 0 ^ s ^ t} 
and 0 {x(s)s £ s £ t}. It is well-known [ 59 ] that the function 

defined by 


\ = exp[ 




^(Xg,s)dzj^(s) 


-j Ch 


j^(Xg,s)h^ CXg,s)ds] 


= exp[ 



h’(x ,s)R ^(s)dz(s) 
s 



h'(x ,s)R ^Cs)h(x ,s)ds] 

s s 


(5.16) 

is a P^) inartingale. Let (j) be a real-valued measurable function 
on^, with the property that e|4)(x^)| < Then we have the folloxvring 
(compare [60]) 


Lemma 5.3.2 The conditional expectation of 4»(x^) given z^ can he 
written as 
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E[«f>(x^) |s ] =* 


t, Ecii'VHllI 




a.s. P 


(5.17) 


dP 

Proof: Since = l-ns it is kno™ [61] that 

- dir X 

O 

.r*. ,1 t, 

EWfcpIz ] E^tl^lzt] 


Using the properties of conditional expectations and the fact that 
is a martingale 5 we can write 

Eo[<i>(Xt)LjU^] = EJe^[({>Cx^.)L^|^j.]|z^} 

= E^{({.(x^)E^[L^|^^]Iz^> 

= Ej<i)(x^)Ljz^} 

! t 1 tZ. 

Similarly, E^EL^jz ] = E^[L^|z ], and the lemma follows. 


In what follows, we shall omit the qualification of almost sure 
equality for conditional expectations. Such qualifications will 
alX'jays be considered understood x^hen conditional expectations are 
involved. 


An application of the exponential formula [59] to the functional 
yields 

= 1 + r Lgh*(Xg,s)R"^(s)dz(s) (5.18) 

We shall recast the numerator of (5.17) into a more convenient fora 

for later calculations. To that end, we make one more assumption: 
rT 

(A. 6) I E|(^(x^)h(x^,t)pdt < 

•U 
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From now oni (A. 6) will be assumed to hold* We then have 


Lemma 5.3.3 


r ^(x^)L^h* (x^,s)R ^(s)dz(s) Iz’^] 

Jn 


-1, 


f' s ' s 


-f 

■^0 


E [(j)Cx )L h' (x ,s) |z^]R ^(s)dz(s) 


(5.19) 


Proof ; An almost identical lemma is proved in Zakai [62]. His proof 
can be adapted to our case without difficulty. 


Lemma 5.3.4 We can then write the numerator of (5.17) as 

.t 


[$(Xj.)L^|z^] = E^[(j)(x^)] + r E^{E^[4>(x^)lXg]Lgh’(Xg,s)|z®}R ^(s)dz(s) 

•^0 


Proof ; Applying (5.18), we can write 

•t 


(5-20) 


Eo[4>(x^)L^|z*^3 =E^W(x^)( 1+ r L^h' (XgjS)R ^(s)dz(s)) Iz’"] 

.'n 


'0 


o‘-^'"t''“ " “o'- f - s- 

•'O 


= E [{|)(x ) I z ]+E [ 


(x^,s)R ^(s)dz(s) |z^] 

(5.21) 

Since under P^, x(t) and z(t) are completely independent, E^[ij)(Xj.) | z^] = 
E^[(j)(x^)]. Combined with (5.19), we get 

E^[(j)(x^)L^i2^] = E^[(f,(x^)]+ p E^[(()(x^)L^h»(x^,s)|z^]R“^(s)dz(s) 

•^0 

Now I E [(j)(x )L h' (x ,s) j z^]R ^(s)dz(s) 




f t4> ’ s) I Z*"vx® ] I z*}R"^ (s) dz (s) 

Jn 
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= r E {E [^(x ) I (sJgjS) [z^}R ^(s)dz(s) 

f 0 0 ^ 

Jr\ 


(5.22) 


t s 

siBce L h' (x ,s) is measurable with respect to the cr-algebra z vx , the 
s s 

smallest cr-algebra containing z and -T ^ t _< s}. Using the 

Markov property of and the independence of x(t) and z(t) under 


we get 


f 

•'0 


E {E [<j)(x ) z^vx^]L h' (x ,s) z^}R ^(s)dz(s) 
o o t s s 


= f E {E [(j)(x ) |x ]L h' (x ,s)|z^}R ^(s)dz(s) 
I O 0 U s s s 

Jf] 


f ^(s)dz(s) 

Jfl 


(5.23) 

The lemma follows from (5.21), (5.22) and (5.23). 

In order to get the representation for the conditional expectation 
®[4*(x^) |z^] , we need to evaluate the denominator of (5.17). This is 
done in the following lemma. 




Lenmia 5 ^3.^5 E^CL^jz^) ^ = 1- ^ E^(L^[z^) ^E[h^ (x^jis) | z^]R ^(s)dv(s) 


o s* 


(5.24) 


-f 


where V(t) - z(t) - I E[h(x 5 s)|z°]ds is the innovations. 

s 






Proof ; Since L = 1 4- I L h* (x as)R (s)dz(s) Lemma 5.3.3 yields 

t I S 5 


r 


E^(L^lz^) =1+1 E^[L^h’(x^,s)|z®]R ^(s)dz(s) 


o^ s ' s 
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Applying the Ito differential rule to the function E^(L^|z^) ^ gives 
dE^(L^|z^)“^ = -E^(L^iz^)"?E^a^h’(x^.,t)|z^)R"^Ct)dz(t) 

+ E^ (L^ I z^) [L^h » (x^ , t) 1 2^ ]R“^ (t ) E^ [L^h’ (x^ , t) I z^ ] dt 

= - E^(L^|z^)"^E[h(x^,t)lz^ldzCt) 

+ E^(Lj.|z^)"^E[h’Cx^,t)lz^3R"^Ct)E[h(x^,t)|z^]dt 

= - E^aj.|z^)"^[h(x^,t)|z*^]R“^(t)dvCt) (5.25) 


Since (5.25) is equivalent to (5.24), the lenima is proved. 


From Lemma 5.3.2, we know that E[tj)(x^) |z^] is given by the product 
of the right hand sides of (5.20) and (5.24). In order not to complicate 
the computations too much, we evaluate one of the terms separately and 
state it as a lemma. 

•t 


Lemma 5.3.6 E 


^(L^|z^) ^ r E^{E^[0 (x^) |Xg]Lgh' (x^,s) |z®}R ^(s)dz(s) 


-f 

•'0 


E® {E^ 1 4) (x^) { Xg ] [h ' (Xg , s ) “h ' (Xg , s ) ] }r“^ (s ) dV (s ) 


r \CLjz^)-^: 

•'O 


E^[^(x^) ]h' (Xg,s)R ^(s)dV(s) 

(5.26) 


Proof; For convenience, define the functions 
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p(t,s) = E^{E^[<i)Cx^) |x^]Lgh* (Xg,s) [z®}R ^(s) 


q(s) = E (L js)R ^(s) 

O b 5 


Using (5«24)» we can write the left hand side of (5.26) as 


; 

Jq 


r 


p(t,s)d 2 (s) - I p(t,s)dz(s) f q(s)dv(s) 

•^0 


(5.27) 


By properties of Ito integrals [63], (5.27) can be rewritten as 


I 


p(t,s)dz(s) - r r p(t,u)dz(u)q(s)dv(s) 

0 •'O •'o 


•t 

P(t,s) I q(u)dV(u)dz(s) - | p(t ,s)q’ (s)ds 


r p(t,s) f 
•'n Jn 


-f 

•'0 




t rt 

E^(Lg|z®) p(t,s)dz(s) - I p(t,s)q* (s)ds 

•'o 


tz jmS 


-f! 


p (t , u) dz (u) q (s) dV (s ) 


'0 •'0 

(where we have used Lemma 5.3.5) 


= r E^(L^|z®) ^p(t,s)dv(s) 

•'o 


-J 7 

•'0 *'0 


p (t , u) dz (u) q (s ) dV (s ) 
(5.28) 
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Now - 


(where 


(using 


t rS 


n 

*'0 *'o 


p (t ,u) dz (u) q (s ) dV(s) 


I {E^[<i>Cs:^)] + { p(t,u)dz(u)}q(s)dv(s) 


Jo 


-'0 


•'0 


) ]q(s)dv(s) 


(* l^s) ]+ r K'^v? K'^’ 

*'0 •'o 


*LJi' (x^,u) |z'^]B. ^(u)dz(u)}q(s)dV(s) 


i 

•'O 


+ 1 ]q(s)dV(s) 


re have used the smoothing property of conditional expectations) 

= ” f fe^[Eo(4>(x^)!xg)]+ ^JEjE^((|)(x^)Ug)|x^]L^h*(s^,u)lz"] 

•^0 Jo 


R ^Cu)dz(u)}q(s)dv(s) 


I 


+ I E [({)(x^)]q(s)dV(s) 


the Markov property of x^,) 


-f 


E® [E^ ((|) (x^ ) I Xg ) ] E® [h ' (Xg , s ) 3 R“^ (s ) dV (s ) 


■*■ I E^(Lg|z®)“^E^[^(x^) ]h’ (x^,s)R"^(s)dV(s) 
Jck 


(5.29) 
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(where we have successively used Lemma 5.3.4, the definition of q(s), 
and Lemma 5.3.2). Combining (5.28) and (5.29) gives the conclusion of 
the lemma. 


We are not^ ready to state the representation theorem for the 
conditional moment functionals. 


Theorem 5.3.1 Suppose 0:*^ R is such that E|<j)(x^)| < and (A.l) - 
(A. 6) hold. Tlien we have the following representation for the con- 
ditional expectation of (() given z*": 


^(x^) = L^t«P(Xj.)] + 


f 

*^0 


[ij>(xj.)ixg]lh' (Xg,s)-h'(x^,s)]}* 


*R~^(s)dv(s) 


(5.30) 


Proof ; From Lemmas 5.3.2 and 5.3.4, we know that 


Kx<.) = E^[^(x^)]E^(L^|z ) 


r E^{E, 

Jn 


{E_^[^(x^) IXglLgh' (x^,s) |z®}« 


R ^(s)dz(s)E (L Jz’^) 


tN-1 


o' t' 


(5.31) 


From Lemma 5.3.5, we conclude that 






Jq 


(X. ) ]E^ (L^ 1 z®)“^ ’ (x , s) R"^ (s) dv (s) 

U O S S ' 


(5.32) 


Substituting (5.32) and (5.26) into (5.31) gives (5.30). The proof is 
completed. 
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given by 


E^[x(t+0)] = E^'*‘®[x(t+0)]+ E®{E Ix(t+0)Ix ]• 

I o s 

*^t+0 




(Xg,s)-h' (Xg,s) ]}R (s)dv(s) 


(5.33) 


Proof: Using Theorem 5*3*1, 


E^[x(t+0)] =E^[xCt+6)]+ r E®{E^[xCt+0)|x^][h*(x^,s)-h'Cx^,s)]}ll"^Cs)dv(s) 

•^0 


I {E^ [ X (t+0 ) I Xg ] [h ' (Xg , s ) -h ’ (Xg , s ) ] } ■ 

n 


•R“^(s)dv(s) 


+ J” ^^■CE^[x(t+0)|x^][h’ (x^,s)-h* (x^js)]}p ^(s)dV(s) 


t+0 


= E^'^®[xCt+0)3+ I E®{E rx(t+0)|x^][h»(x^,s)-h’(x„,s)]}- 

■ O s s s 




t+0 


•R ^Cs)dv(s) 


which is precisely (5.33). 


Remark 5.3*1 Theorem 5.3*1 is a generalization of the corresponding 
representation results for nonlinear filtering of stochastic systems 
without time delays. While the measure transformation techniques 
employed here has been used before in connection with systems without 
delays [58] a [62], the present form of the representation appears to 
be novel even when specialized to systems ^^thout delays. The reason 


for this is that in [58] and [62] , the authors were interested in 
deriving equations for the unnormalized conditional density* They 
did not, therefore, give explicit formulas for the true conditional 
moments. In our case, conditional densities for the process do 
not even make sense. This motivated us to derive the Representation 
Theorem 5.3.1 for conditional moment functionals. We feel that the 
present form of the representation is much more convenient to use 
when we derive stochastic differential equations for conditional 
moment f tmctionals . Furthermore, it shows clearly the role played 
by the infinitesimal generator of the Markov process x^. In the 
next two sections, we shall apply Theorem 5.3.1 to derive stochastic 
differential equations for the nonlinear and linear filtering problems 

5.4 Stochastic Differential Equations for Nonlinear 




Let us first make a few preliminary calculations. Let e > 0 be 


fixed and let h(x- ,s) = E[h(x ,s)(z ], Then 

5 S 








E{E^[<{iCx^^^) |Xg][h(Xg,s)-h(x^,s)]' |z®} 


-R"-^(s)dv(s) 


- r E{E^[^(x^)iXg][h(Xg,s)-h(Xg,s)3’]z®}R ^(s)dv(s) 

•Vl 




h f E{E^ 

Jn 




[h(x ,s)-h(x ,s)r!z^}lJ. *^(s)dvCs) 

5 5 

f t+e ^ _- 

E{E^ [4> (x^+g.)lXg ] [h (Xg , s ) -h (Xg , s) ] ' 1 z® } r“'^ (s) dv (s) 

(5.34) 

We will be primarily interested in the limit as s -»• 0. The last term 
in (5.34) is straightforward to evaluate in that case. The difficult 
calculation is that of evaluating terms of the form 


= EjE^[<|)(x^^)-<J>(x^)ix^vXg]lx^} 

= E^{E^[4>(x^^g.)-4>(x^)ix^]|Xg} 

where we have used the Markov property of x^. The evaluation of 
Ejj[0 (x^^)“^> (x^) I x^] corresponds to the determination of the in- 
finitesimal generator of the Markov process x. . 



Let us assume for the time being that the functional 4» is in 
the domain of the generator, of the Markov process x^. Then it 


is true (see Dyrikin [64]) that 




(5.35) 


In this case, we have the following 


Theorem 5.4.1 Let ^ satisfy the conditions of Theorem 5.3.1. In 

addition, let 4»C2i^) be in the domain of the infinitesimal generator 

of x^ and suppose that 

r e| tj5|.^(x^) ]h(x^,t) j^dt 
•'O 

Then the functional <f)(x^) satisfies the stochastic differential equation 

dkx^.) = E[^(f)(x^)|z^]dt + E[(j)(x^)(h(x^,t)-h(x^,t))Mz^]R“^Ct)dv(t) 

(5.36) 

Proof : We start with (5.34). For simplicity, let e^^Ct) = h(x^, t)-h(x^,t) 

Then 

r E{E^[^(x^^)-^(x^) |xg]e^(s) lz®}R ^(s)dv(s) 

-'o 

= f E{E^{E^[(j)(x^^^)-4>(x^) |x^]|Xg}e^(s) |z®}R ^(s)dv(s) 

•^0 

J rt /-t+e c -1 

E{E { I E [^^(x ) [ X ]du|x }e' (s) I z®>R ■“■(s)dv(s) 

I Of OU-lXu sn 

0 “'t 

rt ^t+S e _1 

“I }duR ■‘■(s)dv(s) 










f t+E rt 

I E{E^[j!^^)Cx^) ix^]e^(s) |z®}R~‘‘’(s)dv(s)du 

-'o 


(5.37) 


using Lemma 5.3.3. 
Next, 


( t+6 

. 


E{E^[(J)(x^_j_^) |x^]e^(s) [z®}E.~^(s)dV(s) 


I E{[<j)(Xg)+ I E^[ja^4)Cx^) |Xg)]e^(s) [z®}R ^(s)dv(s) 


T 


E[(f)( 2 Cg)e^Cs) [z®]R ^(s)dv(s) 


f t+E ^t+E 

I e{e^[^^i(x^) [Xgle^(s) |z®}R (s)dv(s) 


(5.38) 


Finally 




Eofe„[*C*j+P - '('C*t)i*tl 

I. 


E [t^(}>(x ) ]du 
0 u 


rt+e 
“1 ■ 


E[v^<f)(x^) [z^Jdu- 


J t+E 

I E{E^M^<|>(x^) |Xg]e^(s))z®}R Cs)dv(s)du 
*^0 


(5.39) 
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using the representation in Theorem 5.3.1. Adding (5.37) - (5.39), 
we get 


»t+e /-t+e 


f t-re 

E[^(j)(x^) |z'^]du+ I E[<j)(x^)e^(u) jz^]R (u)dv(u) 


/ t+e^t+e 

I E{E^(tj^^(x^) |xg]e^(s) lz®}duE (s)dv(s) 


't "s 


I j E{E^t^<j)(x^)|Xg]e^(s)lz^}R -^(s)dV(s)du 
A A (5.40) 


Another application of Lemma 5.3.3 shows that the last two terms of 
(5.40) add to zero. The proof is completed. 

Theorem 5.4.1 is a generalization to systems T^ith delays, of the 
usual formula for conditional moments of ordinary diffusion processes. 
I'lhile the form of the stochastic differential equation is exactly the 
same as that for diffusion processes, there is a subtle difference. 

In the diffusion process case, x(t) itself is a Markov process. In 
the delay case, while x(t) still satisfies a stochastic differential 
equation, it is no longer a Markov process. This special structure 
of stochastic delay systems is clearly shown in (5.36), where the 
infinitesimal generator of x^ plays a crucial role. Thus, in 
order to apply Theorem 5.4.1, we need to characterize the domain 
of the infinitesimal generator of the Markov process x^. These are 
functionals for which the limit 
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(5.41) 


lim r {E[({)(x , )lx ] - (J)Cx )} 

h ^ 0 CTO t c 

necessarily exists. This immediately rules out functionals of the 
form 4>(x^) = <J>[x(t+0)l for some 9 e (*-r,0). For in this case, 

E[(f)(x^_j_j^) [x^] = E{(j)[xCt+9-Ri)]{x^} 

= <j)[x(t+0+h) ] 

whenever -T 0 + h _< 0. Since the sample path x(t) is not differ- 
entiable, the limit in (5.41) does not exxst. It is not possible 
therefore, to derive a stochastic differential equation for a 
functional of the form ^[x(t+0)] using Theorem 5.4.1. 

Thert! are certain special classes of functionals (j) which are 
in the domain of the generator These are those considered by 
Kushner [54]. We shall simply state these results. 

Case 1: Suppose the functional <|)(x^) = 4>[x(t)], and is twice con- 

tinuously differentiable in its argument, then 

«^|.<|)[xCt) ] = f (x^,t) '<i)^[x(t) ] + |- tr Q(t)^^(x(t)) 

where <j)^ is the n-vector whose i^^ component is g^[x(t) ] . Sub- 

i 

stituting into (5.36) gives 

d$[x(t)] = E^{f (x^,t) ’(fi^[x(t) ]+ j tr Q(t)^^[x(t) ]}dt 

O 

H- E^{(f>[x(t)][h'(x^,t)~h’(x^,t)]}R"^(t)dv(t) 

(5.42) 
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Th-fg is precisely the formula derived in Fujisaki et al., [51] 
using a different technique. We shall denote the operator,;^ 
in this special case by^^. In particulars the conditional mean 
x(t) satisfies 

dxCt) = E^[f (x^,t)]dt + E^{x(t)[h’(x^,t)-h’(x^jt)]>R~^(t)du(t) 

(5.43) 

Equation (5*43) and the formula for the smoothed estimate, (5.33), 
will be useful in discussing the optimal linear filter. 

Case 2 1 Let “ j t|^C9)g[x(t+0) , x(t)]d0 where is continuously 

differentiable on [^T,0] and g is twice continuously differentiable 
in its second argument* Then 

= 4'C0)g[x(t),x(t)]-lJJ(-T)g[x(t“T),x(t)] 

t t 

4'(0)^S[x(t+8),x(t)]d0 

(5.44) 

where ^ is the operator defined in Case 1 and acts on g as a function 
of x(t) only. 


-fi 

-T 


^(0)g[x(t+0),x(t)]d0+ 


J 

•'“T 


Case 3 : Let <f)(x ) =* D[F(x )] where D is a twice continuously dif 

t t 


ferentiable real-valued function, and F(x^) 

t 


c 


ij^(0)g[x(t+0), x(t)]d0 


is the type of functional described in Case 2. Then 




a=F(x^.) 


+ 2 


a=F(Xj,) 
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where 




^0 ^0 

G= I I Sq [x(t+0),x(t)]g- Ix(t+0),x(t)]Q (t)d0dr| 

J_T J~r i,j h 

til 

and denotes partial differentiation of g with respect to the x 
component of the second argument* 

From the above special cases, we can see that basically we need 
t^^ice continuous differentiability of (j) with respect to the dependence 
on x(t) , and Frechet differentiability with respect to the dependence 
on the piece of the trajectory As discussed before, this rules 

out functionals of the form ^[x(t+0)],0 e [-T,0)* Hence for nonlinear 
systems with point delays, any attempt in deriving stochastic differ- 
ential equations for conditional moment functionals will have to face 
the difficulty of functionals not being in the domain of the generator 
of the Markov process For example, as in the multipath communica- 

tion problem mentioned in Chapter 1, if the observation process is of 
the form 

dz(t) = {h^[x(t) ]+h^[x(t-T)]}dt + dv(t) 
then for a twice continuously differentiable cj>[x(t)], we get 
d$[x(t)] E^tx(ty)h^ + 

-|Cx(t))h^(x(t)) - $(xCt))h2(x(t-T))]'R"^Ct)dvCt) 

(5.45) 

If we try to write stochastic differential equations for the unknown 
quantities on the right hand side of (5*45), we see that this cannot 


)(x(t))h2(x(t^ 
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be done for all the unknoiMns, since (j)(x(t))h2(3c(t-T)) does not lie 

in the domain oft^, 0£ course, there are many physical problems 
t 

(for example, radar problems with spread targets [11]) where the 


observations are of the fomn h(x^) 



i|^(0)Htx(t+0),x(t)]d0. 


Moreover, one can approximate point delays by distributed delays of 
the above form* This will allow us to write a stochastic differ- 
ential equation for '^Ix^tTlhO^ However, x-je will then get the 
unkno^<Fnt^^<J>[x(t0^l^^ in our equation for If 4 »(-t) 0, 

]h(x^) will contain a term with point delay (see Case 2 above), 
and we are faced with the same problem as before. In general, if 
the functionals involved are in the domain ofjz^^, i = l,...n, we can 
write n coupled stochastic differential equations involving the 
moment functionals, just as in the diffusion process case. It should 


be clear from the above discussion that this puts rather severe re- 
strictions on the functionals involved. Thus, the study of approx- 
imations to the optimal nonlinear filter for delay system remains an 
important and open problem. 


There is, however, one special case x-zhere the optimal filter can 
be completely specified even when there are point delays in the syscem- 
This is the linear case with Gaussian distributions and will be treated 
next. 


5.5 Optimal Filtering of Linear Stochastic Delay Systems 
We consider the system defined by 
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(5.46) 


dx(t) = a(x^,t)dt + dw(t) 

x(9) = X (0) 0 e [-T,0] 

0 

dz(t) = c(x^,t)dt + dv(t) 


Here a: •g’xfOjT] -> R 

c: ‘^x[0,T] ->■ R 

are continuous linear functionals given by 


a(x^,t) = 


c(Xj.,t) = 



dQA(t,0)x(t+6) 


dgC(t,0)x(t+9) 


(5.47) 


where A(t,0) and C(t,0) satisfy the same conditions as those imposed 

on such functions in section 5.2. Also, we take x to be a Gaussian 

o 

process on [^T,0] with sup Ejx (9) | < By a similar argument to 

0 

the case without delays [56], it is readily seen that the conditional 

distributions of x(t+9), for any 0 e[“T>0], given zCs)^ G^s;£t, 

is Gaussian, For greater clarity in the subsequent exposition^ we 

shall write x(tjt) “ E{x(t)j 2 ^} and S:(t+6|t) = E{x(t-t0 [ 2 ^} , 9 e[-Tj03^ 

Using (5,43), we immediately obtain the following stochastic differ^ 

ential equation for the conditional mean 

rO 

dx(t|t) = I dgA(t5 9)x(t‘{-0 1 t)dt 

+[ E^(x(t)x’(t+0|t))dgC'(t,0) 

•Lt 

0 

x(t| t)x' (t+9| t)dgC* (t,0) ]R (t)dv(t) 

•T 



(5.48) 


Here the innovations process v(t) is given by z(t) — 


/:/: 


dgC {s , 6) x(s+0 j s) ds 


and hence depends, in general, on the smoothed estimates x(s+9js), 
-T£0£O, 0£s5t. Define the "smoothed" conditional error co- 
variance as 

P(t,9,0 = E*^{[xCt+0)-xCt+6|t)][x(t+C)-x(t+e|t)l’} 


Then (5.48) can be rewritten as 


rv 

dx(t{t) = I dgA(t,0)x(t+0 j t)dt 


-1 

+ j P(t,O,0)dQC’(t,0)R (t)dV(t) 


(5.49) 


To evaluate the unknown terms on the right side of (5.49) , we use 
(5.44) to write the smoothed estimate as 

__T 

x(t+0|t) = x(t+6|t+0)+ I E®{x(t+0) [c' (Xg,s)-c' (XgjS) 1)R (s)dv(s) 

•^t+0 


— x(t+0[t+0)+ 



P (s , t+0-s , C) d^c (s , ^ ) ’ (s ) du (s ) 

(5.50) 


for 0 e[-T,0] . 

An inspection of (5.49) and (5.50) shows that the optimal linear filter 
is completely characterized by x(t+0|t), -T 0 ^0, and the "smoothed" 
error covariance function P(t,0,?). It remains only to derive ap- 
propriate equations for P(t,0,C). Since the derivations are quite 
lengthy, we will present them in Appendix B, and state the final 
result here as 
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Theorem 5.5.'l The optimal filter for the system (5.46), (5.47) is 
characterized by the following sets of equations: 

(i) Generation of the condition mean x(t|t): 
rO 

dx(t|t) = I dpjA(t,0)x(t+6| t)dt 


*6' 


.0 

+ P(t,O,0)dgC'(t,0)R ^(t)dv(t) (5.51) 

(ii) Generation of the smoothed estimate x(t+9jt), -T 0 <_ 0: 


,-l. 


x(t+0|t) = x(t+0|t+0)+ 


rr 

‘'t+9*'-T 


P (s,t+0“S,^)d^C(s jC) 

( 5 * 52 ) 


(iii) Generation of the smoothed error covariance PCtjG,^): 

P(t,0,0)= j* P(t,O,0)dQA’(t,9)+ j* dgA(t,6)P(t,0,O) 


-rr 


P(t,O,0)dQC’ (t,0)R"-^(t)d^C(t,C)P(t,^, 


'-f-T 


C5.53) 


'■r 

•'-T 


'^P^(t, 0 ,O)= I P(t, 0 ,?)d^A‘(t,O 


-£/: P(t,0,C)d^C'(t,^)R"^(t)d^C(t,a)P(t,O 

/er c A\ 


(5.54) 


/•O fO _i 

/3 P^(t, 0 , 5 )= - I I p(t, 0 , 3 )dgC'(t,e)R ■'■(t)d^c(t,a)P(t,a, 5 ) 


(5.55) 


0)+Q(t) 


where n is the unit vector in the (1,-1) direction, O is the unit 
vector in the (1,-1,-1) direction, and P„(t,0,O) and P„(t,0,?) are 







the directional derivatives of P(t,0,O) and P(t,6,^) in the directions 
n and a respectively. The initial conditions are given by 

x(0|0) = x^(0), x(6|0) = x^(0) e e[-T,0] 

P(O,0,Q = 1^(6,?) = E{[x^(0)-x^(0)][x^(O-x^ (?)]'} 

(5.56) 

A few comments about Theorem 5.5.1 is in order. Notice the great 
similarity as well as the striking differences in the fom of the 
solution to the filtering problem for linear delay systems with 
Gaussian distributions and the corresponding solution for linear 
systems xv^thout delays. In our case, we need to characterize not only the 
conditional mean 5c(t|t) (as in the non-delay case), but also the 
smoothed estimate x(t-l-9|t), not only the estimation error covariance 
P(t,0,0), but also the smoothed error covariance function P(t,9,0* 

The complexity of the solution is clearly considerably increased when 
there are delays in the system. It is hoped, however, that the results 
presented in this and the next chapter will provide a theoretical 
foundation for the study of implementable filters for delay systems. 

Note that our development only shows that P(t59,^) is continuous 
and has directional derivatives. Of courses, if P(t,0,^) were actually 
differentiable in (t,0,C)j we could rewrite 

“ (at" “ w) 

and S p„(t,6.0) - (1^ - le - y nt.e.C) 

We would then obtain partial differential equations for P(tj6j^). 
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Theorem 5.5.1 completely solves the filtering problem for general 
linear stochastic delay systems. Notice that although functionals of 
the form “ x(t+0), -x < 6 < 0, do not lie in the domain of the 

generator of x^, we are still able to calculate the directional de- 
rivatives of the covariances. This is a consequence of the linearity 
and Gaussian assumptions. 

Let us make a few remarks on the relationship of our work to past 
investigations. The linear filtering problem for systems with only 
point delays was first considered by Kwakemaak [65], whose results 
are similar to those presented in this section. He restricted his 
attention to linear filters and followed the approach of Kalman and 
Bucy [66], and his derivations were formal. In partictilar, he used 
partial derivatives liberally when these were not defined. His 
"proofs”, therefore, were far from satisfactory. Lindquist also 
considered the linear filtering problem in [56]. However, he only 
gave a complicated integral equation for the filter gain, and did 
not derive equations for the covariances. His results are thus in- 
complete and do not display the structure of the filter equations. 

In particular, they are not suitable for the filter stability in- 
vestigations that we pursue in Chapter 6. Recently, Mitter and 
Vinter [67 ] have studied the linear filtering problem from the 
viewpoint of stochastic evolution equations. While they did take 
in account some of the special features of delay systems, they had 
to exclude point delays from their observation model, an unsatis- 
factory restriction from various points of view. Furthermore, 
their derivation of the optimal filtering equations proceeds via a 


- 127 - 


dual control problem. As such, their approach cannot be extended 
to the nonlinear situation. By contrast, our approach enables us 
to allow a very general model for the stochastic delay system under 


consideration, including distributed as well as point delays. Co- 
variance equations have been derived rigorously and the structure 
of the optimal filter is clearly displayed. We feel, therefore, 
that our results on linear filtering are the most complete and 
satisfactory to date. 

In the special case where x^ = 0, a(x^,t) = Ax(t) + Bx(t-T), 
c(x^,t) = Cx(t),Q, R constant matrices, it can be shown by exploiting 
the connection between linear optimal filtering and optimal control 
with quadratic criterion that P(t,0,?) are in fact continuously 
differentiable in (t,0,C). When we compare the solutions to the 
linear optimal control and optimal linear filtering problems in 
Chapter 6 , it will be helpful to use the notation 


p^(t) = p(t,0,0) 

(5.5?: 

p^(t,0) » P(t,0,O) 

(5.5S: 

P2(t,0,C) = P(t,0,?) 

( 5 . 59 : 


In this case the optimal filter is given by the equations 


dx(t|t)=Ax(t|t)+Bx(t-T|t)+P (t)C’R~^[dz(t)-Cx(t|t)dt] 


(5,60 


x(t-T| t)=S(t-T| t-T)+ r P 2 (s,t-x-s,0)C'R'''‘'[dz(s)~Cx(s|s)ds] 

(5.61 




x(0|O) = 0 


-T < 0 < 0 


§-P^(t) = AP^(t)+P^(t)A*-P (t)C'R ^CP (t)+Q+BP (t,-T)+P’(t,-T)B' 
at o a o o o J. X 

(5.62) 

(It “ le) = P^(t,0)[A'-C'R”^CP^Ct)]+P2(t,e,-T)B' (5.63) 

(fe " le " If) ^ -P^(t,9)C'R"V^(t,P (5.64) 

with P (0) = P, (0,0) = P„(O,0,O = 0 -T < 0, ^ < 0 

O L A — — 

P^(t,0) = P^(t) 

P2(t,0,O) = Pj^(t,0) 

P^(t) = P;(t), P2(t,0,C) = P^(t,?,0) (5.65) 

We shall be using these equations extensively in the next chapter. 

Notice that in this special case, x(t|t) depends only on 
x(s|s), t-T ^ s ^ t, and from (5.60) and (5.61), we can obtain 
an explicit delay equation for x(t|t). This is because we have 
no delays in the observations. If c(x^,t) were of the form 
C^x(t) + C^x(t-T), then we would have the following equations 

dx(t| t)=Ax(t| t)+Bx(t-x| t)+[P (t)C'+Pi (t,-T)CnR 

O O X X 

• [dz(t)-C^x(t| t)dt-Cj^x(t-T|t)dt] (5.66) 

x(t-r[ t)=x(t“T| t“X)+ f [P2(s,t-x-s,0)C^+P2(s,t-x-s,-x)Cj^]R 

•4-x 

[dz (s)-C^x(s I s)ds-C^x(s-x| s)ds] (5.67) 
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If we substitute (5.67) into (5.66), we see that x(t|t) depends not 
only on x(s|s), t-T £ s < t, but also on delayed smoothed estimates 
x(s-t[s), t“T ^ s t. The presence of the delayed smoothed 
estimates prevents us from obtaining a delay equation for x(t|t). 

The additional complications which delayed observations introduce 
will be discussed further in section 6.7. 

It is worth pointing out that in the case where = 0, 

S(t+6|t), -X < 0 < 0 , can be expressed as a Wiener integral with 
respect to the observations s(s), 0 ^ s ^ t, i.e. , 

x(t+9|t) = f K(t,0,s)dz (s) (5.1 

Jo 

for some kernel K(t,0jS). To see this, note that x(t) is a Gaussian 
process. Hence the orthogonal projection of x(t4-0) onto the Hilbert 
space spanned by the observations z(s), 0 _< s _< t is the same as the 
conditional mean. Since the orthogonal projections can be expressed 
in the form of the right hand side of (5.66), so can x(t+0[t). This 
fact will be used in Chapter 6. 

So far, we have studied optimal filtering for stochastic delay 
systems on a finite time interval. To make the filtering theory of 
delay systems more complete, we shall study the asymptotic behavior 
and stability properties of the optimal linear filter in the next 
chapter. 


CHAPTER 6 


STABILITY OF LINEAR OPTIMAL FILTERS AND CONTROL SYSTEMS 
6*1 Introduction 

In the previous chapter, we derived optimal filter equations for 
linear and nonlinear stochastic delay differential systems* In this 
chapter, we study the stability of the linear optimal filter for delay 
systems. We shall concern ourselves primarily with stochastic systems 
with delays in the system dynamics, but no delays in the observations. 
Extensions to the general case, including delays in the observations, 
will also be discussed. In proving the stability results, we make 
essential use of the duality between optimal filtering of linear 
stochastic delay systems and optimal control of linear delay systems 
with quadratic cost. We shall therefore begin the development with a 
summary of the known results for optimal control of linear delay systems 
with quadratic cost, and give some new results on the asymptotic sta- 
bility of the optimal closed- loop control system. This brings into 
focus the role played by the concepts of stabilizability and observa- 
bility* Next, we give a duality theorem which establishes the connection 
between optimal linear filtering and optimal control of linear delay 
systems. This enables us to identify the gains of the optimal linear 
filter with the gains of a "dual" control system in an appropriate 
TAfay. We can therefore exploit the known convergence of the control 
gains to infer the convergence of the filter gains. The stability of 
the optimal filter is then established by constructing a Lyapunov 
functional. Once the stability of the optimal linear filter has been 
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established, it is relatively straightforward to prove the asymptotic 
stability of the optimal linear stochastic control system, drawing 
upon known results on the separation theorem for stochastic delay 
systems [56], [68]. 

6.2 Optimal Control of Linear Delay Systems with 
Quadratic Cost 

The problem of optimal control of linear delay systems with 
quadratic cost has received considerable attention in recent years. 
Various methods [69] - [73] have been devised for solving this problem. 
The simplest version of the problem can be formulated as follows i 

We are given a linear constant delay system of the form 

= Ax(t) + Bx(t-X) + Cu(t) 
dt 

x(6) = ((>(0) 0 e [-T, 0] (6.1) 

Our choice of function space mil be although basically the same 

2 

results hold for other choices, such asM [29] , [73]. The admis- 
sible control set U is the set of R“-valued functions on [0, T]. 

The cost functional is given by 

= I [x’(t)Qx(t) + u' (t)Ru(t) ]dt 

where Q and R are symmetric matrices of appropriate dimensions, Q ^ 0, 

R > 0. The objective is to find a control u e U such that J^(u, (()) is 
minimized. The cases when T is finite or infinite have been studied. 

We shall briefly survey the knoxjn results and then give some extensions 
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We start with the case when T < “. 


Then it is well-knoxm [69] - 


[73] that the optimal control can be expressed in feedback form by 


u*(t) = -r“^C'K (t)x(t) - r“^C' 
o 



K^(t,e)x(t+9)de (6,2) 


The feedback gains satisfy the following coupled set of partial dif- 
ferential equations 


^ K (t) = -A'K (t) - K (t)A + K (t)CR ^C*K (t) - Q - K (t,0) - K' (t,0) 
at o o o o o -LX 

(6.3) 

(it " le) = -[A’-K^(t)CR"^C']K^Ct,0) - K2(t,O,0) 

(6.4) 

(it “ fe “ If) " Kj^(t,0)CR"^C'K^(t,O (6.5) 

The boundary conditions are 


K^(T) = K^(T,0) = K2(T,0,^) - 0 ~T < 0, C £ 0 

K^(t,-x) = K^(t)B ) (6.6) 

K2(t,-T,0) = B'K^(t,0) I 

Furthermore, 

K^Ct) = K*(t), K2(t,0,e) = K^(t.?,0) (6.7) 
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The optimal cost can be expressed as 


“ (j>'(0)K^(0)<i)(0) + J (f)'(o)Kj^(o,e)(j)(0)de 

-T 

+ J ())’ (0)K^(O,6)(|)(O)d0 

“T 

-n 0 

(f>»(e)K^(o,0,O(j)(Oded? (6,8) 

_T ~T 

The integral term in Eq. (6.2) represents the effects of the delay on 
the optimal control. Thus the optimal control is seen to be given by 
p linear map K operating on the complete state x^. Similarly, the 
optimal cost can be thought of as a generalized quadratic form. 

We now discuss the infinite^-time control problem, i.e., T = 

For the control problem to be meaningful, we need some condition to 
ensure that the optimal cost will be finite. It turns out that the 
system-theoretic concepts of stabilizability and observability intro- 
duced in Chapter 2 play a crucial role* The following is the basic 
result on the convergence of the optimal control law and control 
gains for the infinite-time problem [32], [72]. 

Proposition 6.2*1 Assume that system (6.1) is stabilizable. Then 
the gains K^(t), K^(t,0) and K2(t,0,^), for t < T, converge uniformly 
in t to K^, K^(0)^ and K2(0,C) respectively as T The optimal 

control law for the infinite-time control problem with quadratic cost 
is given by 
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u*(t) = -R“^C’K^x(t) - J R~^C*K^(0)x(t+6)de (6.10) 

-T 

where K 2 ( 0 >^) satisfy the following set of equations 

A’K + K A-K CR“^C’K +Q+K' (0)+K' (0) = 0 (6.11) 

o o o o 1 1 

Iq K^(0) = [A’-K^CR"^C*]K^(9) -*• K2(O,0) (6.12) 

(|g +|^j K2(0,O = -K[(0)CR"^C'K^(O (6.13) 

with boundary conditions 

K (-T) = KB 
X o 

(6.14) 

K2(0, -t) = K^(0)B 
and symmetry conditions 

= K’ , K2(0.O = K^(C,0) (6.15) 


The optimal cost is given by 


Jj(4>) = (j)'(0)K *(0) + 

VM O 



<()'(O)K^(0)(i)(e)d9 



-T “T 



(j)'(0)K2(9,?)4>C?)d6d? 


(6.16) 


Asymptotic stability of the closed-loop system have also been 
studied previously by Datko [72], and Delfour et al. , [32j. In both 
papers, the assumption that Q > 0 was made. Here we relax the assumption 
to observability. 
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Let Q = H*H and let x(t) denote the optimal trajectory corres- 
ponding to the optimal control u*(t), i.e., x(t) satisfies 

-1 —1 
x(t) = (A-CR C’K^)x(t) + Bx(t-T)- I CR C'Kj^(0)x(t+0)de 

-T 

(6.17) 

We then have 


Theorem 6.2 »1 Assume (A,B,C) is stabiiizable and (A,B,H) is observable. 
Then the optimal control for the infinite time problem is given by (6.10) 
and the optimal closed-loop system (6.17) is asymptotically stable. 


Proof ; The first part of the theorem is just Proposition 6.2.1. To 
prove asymptotic stability of the closed-loop system, consider the 
Lyapunov functional 

V(x^.) = x' (t)K^x(t) + j x' (t)Kj^(0)x(t+6)d0 

-T 





x’ (t+0)K^(0)x(t)d0 



x’ (t+ 0 )K 2 ( 6 ,?)x(t+^)d 6 dC 


(6.18) 


This, however, precisely corresponds to the optimal cost for the infinite- 
time problem starting at time t with initial function x^. Thus we can 
write 

CO 

V(x^) = r [x*(s)Qx(s) + u*' (s)Ru*(s) ]ds 
t 

V(x^) is clearly nonnegative. We claim that V(x^) = 0 implies x(t) =0: 
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V(x^) = 0 if and only if 
u(s)=0 

and x’(s)Qx(s) = x' (s)H'Hx(s) = 0 s 

By observability of this implies 

x(s) =0 s ^ t 

Let V((j)) < Si}. Then for some > 0, x^ e implies 

|x(t)I 5. K for some nonnegative constant K. Since V(x^) ^ 0 and 

V(x^) = ~[x’(t)Qx(t) + u*’ (t)Ru*(t) 3 £0 we can apply the invariance 
principle for functional differential equations [74] to conclude 
that the solutions of (6.17) x^ tends to M, the largest invariant set 
contained in the set 

S = {(J>: VCc()) =■ 0} 

Since V(x^) = -[x'Ct)Qx(t) + u*' (t)Ru*(t) ] 

V(x^) = 0 / Hx(t) = 0 

By observability, the largest invariant set M in S are those solutions of 
(6.17) for which |x(t)| = 0 for - » < t < ". Hence M = {O}, and the 
asymptotic stability of (6.17) is proved. 

Remark 6.2.1 It is known that asymptotic stability of autonomous 
linear functional differential equations is equivalent to exponential 
stability [2]. Hence the optimal closed-loop system for the infinite- 
time problem is globally asymptotically stable under the conditions of 
stabilizafaility and observability. 


- 137 - 


Corollary 6.2.1 Assume (A,B»C) is stabilizable and Q > 0. Then the 

optimal closed-loop system (6.17) is asymptotically stable. Since Q > 0 
1/2 

implies Q = H > 0, we immediately have that (A,B,H) is observable and 
we may invoke Theorem 6.2.1. 

Remark 6.2.2 Corollary 6.2.1 was proved earlier by Datko [72] and 
Delfour et al. , [32] using somewhat different techniques. 

Corollary 6.2.2 If (A,B,C) is stabilizable and (A,B,H) is strongly 
observable, then (6.17) is asymptotically stable. 

Proof I Follows from Theorem 6.2.1 since strong observability implies 
observability . 

In proving asymptotic stability of the optimal filter, the same 
basic technique will be used; construction of a suitable Lyapunov 
functional and an appeal to the invariance principle. The system- 
theoretic concepts of detectability and controllability are exploited 
there in a natural way. Before proceeding on that route, we shall 
first study the duality between optimal estimation and control for 
linear delay systems. 


6.3 Duality Between Estimation and Control for Linear 
Delay Systems 

In this section, we extend the duality principle between estimation 
and control for linear ordinary differential systems to linear delay 
differential systems. We shall exploit this duality in proving the 
convergence of the optimal filter gains in the next section. Similar 
results on duality are given by Lindquist [75], and Mitter and Vinter [67]. 
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Consider the optimal filtering problem (in the minimum mean 


square error sense) for the stochastic delay system 
dx(t) = [Ax(t) + Bx(t-T)]dt + dw(t) 

x(0) =0 0 < 0 (6.19) 

dz(t) = [C x(t) + C x(t-T)]dt + dv(t) (6.20) 

o i. 


where w(t) and v(t) are independent Wiener processes with cov[w(t); w(s)] 

QminCtjS), cov[v(t); v(s)] = RminCtjs), Q ^ 0^ and R > 0, We know from 

Chapter 5 that for any vector b, the optimal estimate for b^x(t) can be 

expressed as I k'(s)dz(s) for some function k (s) . We define the 
•6 ^ t 

"dual" control problem to the optimal filtering problem to be: 


minimize J^(b,u) 
subject to the constraint 



ty’ (t)Qy(t)+u' (t)Ru(t) ]dt 


( 6 . 21 ) 


y(t) = -A'y(t)-B'y (t+T)“C^u(t)“C^u(t+T) 


( 6 , 22 ) 


with the boundary conditions 
y(T) = b 

y(s) = 0 s > T 

u(s) = 0 s > T 

As in section 6.2, the admissible controls are L 2 functions on [0, T]. 


The following duality theorem holds. 


Theorem 6.3.1 Consider the optimal estimation of the quantity b'x(T) 
in the system (6.19) - (6.20). Let the optimal estimate for b'x(T) be 
b'x(T|T) and let the optimal control for the dual problem (6.21) - (6.22) 
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(6.23) 


be Uy. Thenb'x(T|T) is related to by 

T 

b'x(T|T) = -J u^Cs)dz(s) 


Proof ; Consider the function 

.?{f(y,x,t) =y’(t)x(t) + / y' (s+T)Bx(s)ds 

t-T 

Inhere y(t) and x(t) are solutions of (6.22) and (6.19) respectively. 

Applying Ito's differential rule, we see that 

d.54f(y,x,t) = ~[y' (t)A+y’ (t+T)B+u* (t)C^+u* (t+T)Cj^]x(t)dt 

+ y ’ (t) {lAx(t)+Bx(t-T) ]dt+dw(t) } 

+ y ' (t+T)Bx(t)dt-y' (t)Bx(t-T)dt 

= -[u’(t)C x(t)+u' (t+t)C-x(t)]dt+y’ (t)dw(t) 
o 1 

= -u' (t) [dz(t)-Cj^x(t-T)dt-dv(t) ]-u* (t+T)Cj^x(t)dt+y* (t)dw(t) 
In view of the boundary conditions for (6.19) and (6.22), we obtain 
,5{f(y,x,T) -.3«f(y,x,0) 

= y’ (T)x(T) = b'x(T) 

T T 

= - r u'(t)dz(t) + r u'(t)dv(t) 

1 ) *{) 

-T _T 

+ 1 y*(t)dw(t) + j u* (t)Cj^x(t-T)dt - | u’ (t+T)Cj^x(t)dt 

*0 ■{) *{) 


T T T 

= - r u’(t)dz(t) + r u*(t)dv(t) + r y*(t)dw(t) 

*0 1) I) 
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Hence 


E{b*x(T) + r u’(t)dz(t)}^ = f [y'Ct)Qy(t) + u' (t)Ru(t) ]dt 
*0 *{) 


Minimizing the mean square estimation error is thus the same problem as 
the dual optimal control problem, and the theorem follows* 


Remark 6,3*1 The duality theorem can be extended to more general delay 
systems such as those with multiple or distributed delays* However, as 
we shall not need such generality in subsequent developments, we shall 
not pursue them here* The restriction of x(6) “ 0, for 0 _< 0, is 
imposed because we t-jant the dual control problem to be one whose 
solution is known explicitly* If we allow a zero-mean random initial 
function with E[x^ (G)x^(^) ] = "T £ 9, 5 £ 0, similar 

arguments show that the dual control problem is to minimize 

J^(b,u) = yM0)5:^(0,0)y(0) + I y’ (9+T)BE^(0,O)d0y(O) 

-T 


•b 



’(0)E (O,0)B'y(0+T)dG4- 
o 



^(0+T)BE^Ce,4)B^y(e^-T)d0dC 


+ 



y* Ct)Qy(t)dt + 



u* (t)Ru(t)dt 


In order to use the duality theorem, we would have to first solve the 
control problem for the cost J^(b,u)* These are additional complications 
which do not require any new concepts* To avoid obscuring the main 
points of our development, we have opted to concentrate on the case 
x(0) =0, 0 <_ 0. 
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We will now restrict our attention to the case where = 0. 
Denoting by C, the system now under consideration is 


dx(t) = [Ax(t) + Bx(t“T)]dt + dw(t) 


x(6) = 0 


(6.24) 


-T < e < 0 


dz(t) = Cx(t)dt + dv(t) 


(6.25) 


The case where # 0 will be discussed at the end of this chapter 
since additional difficulties arise. We first recall from Chapter 5 
the optimal filter equations for the system defined by (6.24) - (6.25). 


dx(t[t) = [A-P^(t)C’R C]x(t|t) + Bx(t-x|t-T) 


-B I P2(t,-T,9)C’R ’^Cx(t-f9|t+0)de + P^(t)C'R dz(t) 


+B f P2(s,t“T-s,0)C’R ^dz(s)dt 


(6.26) 


x(0| 0) = 0 


-T < 0 < 0 


By the variation of constants formula from Chapter 5, we can write the 
solution to (6.26) as 


T 

x(t|t) = J $(T,s)P^(s)C'R"^dz(s) 


f(T,s)B y* 


P 2 (u,S“T-u, 0 )C’R ^dz(u)ds 


(6.27) 


where $(t,s) is the fundamental matrix associated x?ith (6.26), and 
satisfies the matrix differential equation 
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^ " tA-P^(t)C'R ^C]$(t,s) + B$(t-T,s) 


-B P2(t,-T,0)C'R C®(t+0,s)d0 


(6.28) 


§(s,s) = I, $(t,s) =0 t < s 


Applying a Fubini-type theorem for Lebesgue and Wiener integrals to 
(6.27) [50], we can write 

"1 

j 0(T,s)B i P 2 (u,s-T-u, 0)C’R dz(u)ds 


.T „u+T 


/ J 

-X U 


$(T,s)Bp 2 (u,s~T“U, 0 )G*R dsdz(u) 


T /T 


U 


(T ,u+0) BP (u, 6~t) C ’ R"^d0dz (u) 


using the definitions of P2(t,9,C) and Pj^(t,0), and the fact that z(u) = 0 


u < 0. Hence 


:(t1t) = J {$(T,s)P^(s)C*r“V j $(T,,s+0)BP^(s,0-T)C’R"^d0}dz(s) 


(6.29) 


This gives an explicit representation of x(t|t) in the form 


:(tIt) = / 


K(T,s)dz (s) 


On the other hand, Theorem 6*3^1 gives a representation of b'x(T|x) 


in the form 



(6.30) 


» 


u^(s)dz(s) 

We can therefore compare the representations in (6.29) and (6.30) and 
identify the control and filter gains appropriately. This \<ri31 enable 
us to exploit the known convergence properties of the optimal control 
gains. We shall do this in the next section and also prove the asymptotic 
stability of the optimal linear filter. 

6.4 Stability of the Optimal Filter for Linear Systems with 

Delays in the Dynamics 

In this section, we shall put together the results developed in 
the previous sections and prove the asymptotic stability of the optimal 
filter. It is worth noting here that our proof of the convergence of 
the filter gains is rather indirect , being based on the convergence of 
the optimal control gains for the dual control problem. However, it is 
not obvious how a direct argument can be used. Indeed, variational 
arguments for proving convergence of the control gains do not as yet 
have a probabilistic interpretation in the filtering context for delay 
systems. At the present state of our knowledge, it appears that the 
indirect approach has to be adopted. 

We begin now our investigations on the stability of the optimal 
filter. We first adapt the solution of the optimal control problem 
for linear delay systems with quadratic cost to solve the dual control 
problem posed in Eq. (6.21) - (6.22) in the previous section. Through- 
out this section, we will only consider the case where = 0, = C. 
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We make the change of variables s = T-t (s will be defined this 
way tintil further notice). Then Eq. (6.22) becomes 

- ~ y(T“S) = -A’y(T-s)-B'y(T-s+T)-C*u(T-s) (6.31) 

as 

Define y(s) = y(T-s) = y(t) 
u(s) = u(T-s) = u(t) 

(6.31) becomes 


l^y(s) = A’y(s)+B’y(s-T)+C’u(s) 


y(0) = b, y(0) =0, -t < 6 < 0 


(6.32) 


(Although the initial function here is not continuous, the results in 
section 6.2 can be readily extended to cover this case as well [35].) 


The cost functional Jj(b,u) can be written as 


J^(b,u) = J* [y(s)’Qy(s) + u' (s)Ru(s) ]ds 


(6.33) 


The dual control problem has now been cast into the standard form, and 
the results of section 6^.2 can be applied. We can therefore state that 
the optimal control u*(s) is given by 

_ 1 _ _ .0 

u*(s) = -R ■**CK (s)y(s)- I R CK (s ,6)y (s+6)d0 
o J o 

-T 


(6.34) 


where K^(s) = -K^(s)A’-AK^(s) + K^(s)C’R~^CK^(s)-Q-K^(s,0)“K[(s,0) 

(6.35) 
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(fc " le) = -[A-K^(s)C'r“^C]K^(s,0)-K2(s,O,0) (6.36) 


f3 9 9 \ “ 


l9s 90 9^/ ~ K^(sj0)C'R CK^(Sj^) 


(6.37) 


with boundary conditions 


K^(T) = K^(T,0) = K2(T,0,^) =0 , -T < 6, C < 0 


K (s,-T) = K (s)B’ 
X o 


(6.38) 


K2(s,“T, 0) = BK^(s,0) 


and symmetry conditions 


Ko(s) = r(s), (3,0,0 = K’(s,e,0) 


(6.39) 


The closed-loop optimal system is given by 


= [A'-C*R~^CK^(s)]y(s) + B'y(s-T) 


-1 - 

-I C'R •^CKj^(s,0)y(s+0)d0 

‘-T 


(6.40) 


Using the variation of constants formula and the fact that y(s) = 0 
-T ^ s <0, we get that 

y(s) = $(s,0)b (6.41) 

where $(s,r) is the fundamental matrix associated with (6.40). Now 
define 

K^Ct) = K^^Cs) = K^(T-t) 


Kj^(t,0) = K^(s,-0) = Kj^(T-t,-0) 
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The folloxijing lemma is now immediate and summarizes the solution of 
the dual control problem. 


Lemma 


given 


where 


with 


and 


6.4.1 The optimal control for the problem (6.21) - (6.22) 
by 

u*(t) = -R“^CIC^(t)y(t)- r R“^CKj^(t,-6)y(t-e)de 

ic (t) = AK (t) + K (t)A’-K (t)C'R’'^CK (t)+Q-fK, (t,0) + K. 
O O o o o ± 


(If " to) = [A-K^(t)C'R ^C]Kj^(t, 0 ) + K 2 (t,O, 0 ) 

(fe - le - ll) = -Ki(t.9)c'irH(t.C) 

K (0) = K, (0,0) = K„(0,9,e) = 0 O<0, C£T 

0 X i 

K (t,T) = K (t)B’ 

1 o 

K2(t,T,^) = BK3^(t,0 

= K'(t), K„(t,6,g) = Kl(t,?,0) 
o o z z 


is 

(6.42) 

[Ct,0) 

(6.4.-’) 

(6.44) 

(6.45) 

(6.46) 


Furthermore, the optimal closed-loop system satisfies 


7-y(t) = -[A’-C’R ^CK (t)]y(t) - B*y(t+r) 
at o 

T 

+ r C’R“^CK^(t,0)y(t+0)d0 

*{) 

y(T) = b 

y(t) =0 t > T (6.47) 

Let the fundamental matrix associated with (6.47) be Y(t,r). We 
then have the follomng: 

Lemma 6.4.2 Y(t,r) is related to the matrix $(s,r) introduced in 
(6,41) by > 

Y(tjT-r) = $(s,r) (6.48) 


Proof ; By straightforaard verification, the above to^o matrices satisfy 
the same delay equation as well as boundary conditions. Uniqueness then 
yields (6.48). 


Combining Lemmas 6,4.1 and 6.4.2 and using (6.41), we obtain an 
alternative formula for the optimal control 

-1 -1 

u*(t) = -tR CK^(t)Y(t,T) + I R GKj^(t,-0)Y(t-0,T)d01b 


,“1. 


-T 

.T 


i 


- 1 , 


= -[R CK (t)Y(t,T) + I R CK. (t,0)Y(t+0,T)d0]b 


(6.49) 


Substituting (6.49) into (6.30) gives 


b'x(TiT) =6'!^^ Y’ (t,T)K^(t)C'R“^dz(t) 

T X ) 

+ f r Y(t+0,T)'K»(t,0)C'R"^d0d2(t)| 

''n •'n ' f 


0 
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Since b is arbitrary, we obtain 


Lemma 6.4«3 The optimal estimate x(T|t) for the stochastic delay 
system (6.24) - (6,25) can be written as 

x(tIt) = I {Y’ (t,T)K^(t)C'R"V f Y(t+0,T)'K^(t,0)C'R”^d0)dz(t) 

■b ° *^0 

(6.50) 

We are now in a position to compare (6.50) and (6.29). First note 
that they are of exactly the same form, with P^, BP^ occurring at 
the same positions as Y, K^, and One suspects therefore, that 

^(T,t) ° Y(t,T)' 

P (t) = K (t) 
o o 

and BP^(t,0-T) = K[(t,0) 

We proceed to confirm this. Define 

= BP2(t,0-T,C-T)B' 

P3^(t,0) = P]^(t,e-T)B* 

P^(t) = P^(t) 

and recall that P^Ct), P^(t,9), and satisfy (5.62) - (5.65), 

Then it is straightforxjard to verify that P^(t), P-|^(t,0), and 
satisfy exactly the same equations and boundary conditions as K^(t), 
Kj^(t,6), and K2Ct,6,0. By uniqueness of the optimal control and 
optimal filter, we may identify 


-149 


( 6 . 54 ) 


. 50 - 



P^Ct) =-• K (t) 
o o 

Pj^(t,0-T)B’ = K^Ct,0) 

BP2(t,0-T,C-T)B* = K^Ct.e.Q 

01ti.T , O<0,?£T 

The fundamental matrix ^'(tjr) associated with the optimal filter (6.26) 
is related to that of the optimal control system (6.47) by 
<S(t,r) = Y(r,t)’ 

We are now ready to prove asymptotic stability of the optimal 
filter for (6.19) - (6.20). Let Q = H’H. 

Theorem 6.4.1 Suppose (A,B,C) is detectable and (A,B,H') is con- 
trollable. Then the gains of the optimal filter converge, and the 
steady state optimal filter is asymptotically stable. 

Proof ; From the results of Chapter 2, vte know that our hypothesis 

implies that (A',B*,C*) is stabilizable and (A’,B',H) is observable. 

By stabilizability , Proposition 6.2.1 shows that the optimal gains 

for the dual control problem K^(s) , K^(s,0) and ¥ 2 ( 3 , 0 ,^) converge to 

matrices K^, K-j^(9) and K 2 ( 6 s?) respectively as T The definitions 

of Kg(t), K^(t,0) and K 2 (t, 6 , 5 ) now imply that these converge to K^, 

Kj^(0), Kg (0,0 respectively as t -»■ By Lemma 6.4.4, we can conclude that 

P (t) - 9 - P 
0 0 

BPj^(t, 0 ) ^ as t -> «> 

BP2(t,0,OB’ -9- BP2(0,?)B' 
where (see also section 6 , 2 ) 
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(6.57) 


AP +P A'-P C*R CP +Q+BP^ (”T)+P* (-T)B' “ 0 
o o o o X 1 


|^BPj^( 8) = -BP^(0)[A'-C’R“^CP^]-BP2(e,-T)B’ 


(6.58; 


\le ■^If) " BP^(e)C'R"^CP[(C)B' 


( 6 . 59 ; 


Pl(0) = P^ 


P2(e,o) = P3^(0) 


p^ = ?• p„(0,O = p;(?,0) 

o o z z 


(6.60 


Stability of the steady state filter is then governed by the stability 
of the equation 

4 - x(t) = [A-P C’R"^C)x(t) + Bx(t-T) 

at o 


-B ( P„(-t, 0)C’R Cx(t+0)d0 


Using Lemma 6.4.4 and (6.47), we see that the adjoint to the undriven 
steady-state filter equation (6.61) is given by 

l^y(t) = -[A’-C’R"^CP^]y(t) - B'y(t+T) 


j CR'’^CP^(0“T)B’y(t+0)d0 


Let s = -t, y(s) = y(-s) = y(t). Then 


l^y(s) = (A-C’R“^GP^)y(s) + B*y(s-T) 


- J C'R"^CPj^(8-T)B’y(s-0)d0 
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We prove that (6*63) is asymptotically stable. Introduce the 


Lyapunov functional 


V( 7 g) = y'(s)P^y(s) + y'(s) r P^(0-T)B'y(s-e)d0 

*0 


T T T 

J ? (s-e)BP^(e-T)7(s)de h- II y’(s-0)BP2(6-T, 5 -t)B 


0 0 


•y(s“C)d0dC 


(6.64) 


Some standard computations show that 


v(y ) = - y'Cs)Qy(s) + [r"^CP y(s) + Tr ^CP’ (0-T)B'y(s -0)10] 'R 
s ( o 1 


• [R ^CP^y(s) + J* R ^CP^(0-T)B'y(s-0)d0]| (6.65) 


Comparing (6*64) and (6*65) with the situation in Theorem 6*2, 1, we see 
that they are completely analogous. By the same arguments and the ob- 
servability of (A*,B’,H )5 we conclude that (6,63) is asymptotically 
stable. Let the fundamental matrices associated with (6.61), (6.62) 

and (6.63) be $ (t^r), Y (t.r) and Y (t,r) respectively. Then it is 
o o o 

easy to see that Y^(t,r) = Y^(-t,-r). It is known [2] that asymptotic 
stability of (6.63) is equivalent to the existence of constants a > 0, 

M ^ 1 such that 

||Y^(s,r)|j 

Thus ||Y^(t,r)I| = iiY^(-t,-r)l! < 


Since $^(t,r) = Y^(rjt), [ $^(t,r)|| _< Me 


-a(t“r) 
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The proof of the theorem is now complete. 


T^omerk 6.4.1 It should be pointed out that the convergence of the 
filter gains P^Ct), BP^(t,0) and BP2(t,0,C)B' do not provide as yet 
the convergence of the "smoothed" error covariance p2(t,0,O. Of 
course, if B is nonsingular, BP2(t,0,C)B’ converging will imply that 
P2(t,0,C). It is interesting to note that the nonsingularity of B 
seems to be related to many questions in the theory of delay equations. 
For example, it is related to the solution semigroup T(t) being one- 
to-one [2], the completeness of exponential solutions [76], and the 
convergence of projection series [77]. 

Instead of identifying P^(t), BP^(t,0), and BP2(t,0,?)B* with the 
corresponding control gains as in Lemma 6.4.4, one can directly Identify 
P (t), P, (t,0), and P„(t,0,?) with the Fredholm kernel P (t,0,C) intro- 
duced by Manitius in the optimal control with quadratic cost [71]. 

The details are straightfor^<rard and are left as an amusement for the 
reader. 

6.5 Convergence of Estimation Error Covariance Operator 

The results of the last section show that under suitable detect- 
ability and controllability conditions, the estimation error covariance 
P^(t) converges. Our objective in this section is to prove the con- 
vergence of the error covariance operator for the steady state optimal 
filter. If we express the estimation error e(t|t) = x(t) - x(t| t) in 
terms of a stochastic delay equation, its solutions define a trajectoiry 
in the space of continuous functions. By the error covariance operator 
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we mean the function 


E(t,e,?) = E{e(t+9lt+0)e' (t+Clt+€)}» -T < 6, ? < 0. 

Of course S(t,0,0) = P (t) . However, E(t,9,0) P_(t,9), and 

E(t,9,C) P2(t,0,?). This is because E(t,9,0) and E(t,9,C) are co- 

variances of the filtering estimation error, whereas P^(t,9) and 
P2(t,0,?) are covariances of the smoothing estimation error. While 
we can show that S(t,9,0 converges to a steady-state operator 
S^(0,C), we have not been able to show the convergence of P(t,9,C)> 
unless B is invertible (see Remark 6.4.1). 


It is easy to see that e(t|t) satisfies the stochastic differ- 
ential equation 

de(t|t) = [CA-P^C'R"^C)e(t|t) + Be(t-T|t-x) ]dt 





BP2(-T,9)C*R”^Ce(t+9|t+0)d9dt + dw(t) 


- P^C'R ^dv(t)-B J p2(-T,9)C»R"^dQv(t+9)dt 
-T 

( 6 . 66 ) 

Note that for any fixed t, v(t+0) 9e[-T,0] is a BrOTTOian motion, and 
hence the last stochastic integral is well-defined. Again, let $^(t,s) 
be the fundamental matrix associated with (6.66). Since (6.66) is an 
autonomous equation 4>Q(t,s) = $^(t-s,0) = $j^(t-s). Then the variation 
of constants formula gives 
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.(tit) . / 




,t fc 

$^(t-s)dw(s) “ I ^>j^(t-s)P^C’R "^dvCs) 

0 *^0 


0 




$^(t-s)B P 2 (-t, 9 )C*R dgv(s+9)ds 

“T 


- f f 

*'0 “T 

(6.67) 

Using a Fubini-type theorem in Doob (50] » the last term in (6.67) can 


be written as 
-t 


J ^l(t-s) 


0 


y P2(-T,a-s)dv(0)ds 


s-T 


-t -min(a+T,t) 
* 1 ) 


$^(t-s)P2(-T,a-s)dsdv(a) 


-t min(t-a,T) 

I I $3^(t-a-e)P2(-T,-g)dgdv(a) (6.68) 


where we have used v(t) =0, t ^ 0. For convenience, we define 


^min(t-s,T) 


L(t-s) = $j^(t-s)P^G’R~^+ J $^(t-s-B)P 2 ("T,-g)d 8 


If we now compute the characteristic functional of the ^-valued random 
variable ” e(t+9| t+0)) ,then by exactly the same arguments 
as in Chapter 5, we get, for y e^*, 

= E expi < y,e^j^> 


E exp<i 


J* dy’(r)j^J* $j^(t+r-s)dw(s)“ ^ L(t+r-s)dv(s^| 
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using ^^(t-s) =0, t ^ B. 


Since for each fixed t, the exponent is a sum of Gaussian random 
variables, and w(t) and v(t) are completely Independent, we can evaluate 
the expectation as 


exp - 


1 ( 

— I j j dy^(r) J $j^(t+r-s)Q$^(t+a-s)dsdy (a) 


-T -T 


.0 0 

■f I I dy'(r) 

-T -T 


r 


L (t+r-s ) RL ' (t-KJ-s ) dsdy (o)| (6.69) 


Therefore, the covariance operator is given by 

?:(t,0,C) = J $3^(t+0-s)Q$[(t+5-s)ds 


+ r L(t+e-s)RL’ (t+€-s)ds 

4l 


j* [$j^(0+s)Q$j^(C+s) + L(0+s)RL' (C+s)lds (6.70) 


The convergence of the covariance operator S(t,0,?) is given by 


Theorem 6.5.1 The covariance operator 2(t,0,C) for the optimal 
estimation error converges pointwise in 0 and ^ to a stationary 
covariance operator 2^j(0,5)j with 


2„C0,C) = f $^(&fs)Q$J^(C+s)ds 
0 

CO 

^ /[«>, (6+s)P C'r“^+ 

0 1 0 

•R[$^(C+s)P^C'r"-^ 

J minC?4-SsT) 

$^(^+s-B)P2("T,»B)dg]ds (6.71) 

Proof: From (6.70), it is clear that ZCt^Gj^) is a symmetric operator, 
luonotonically Increasing in t. (See Remark 6*5.1 following the proof.) 
Since the optimal filter is asymptotically stable, ||$^(t“s)|| £ Me 
for some a > 0, M ^ 1. We can therefore make the estimate 
^t 

[| $^(0+s)Q?>^(C+s)ds|! 

*0 

< 1 |$3^(6+s)Q$^(C+s)|lds 

*0 

< ! IqI 1 e~“^^-^®^ds 

= ( l -^) 

which converges as t By a similar argument, the second term in 

(6.70) is also uniformly bounded in t for each fixed 0 and We can 
now apply a theorem on the convergence of monotone symmetric operators 


/ 


■min ( 0+s , t) 


o^(e+s-3)P2(-T, 


-3)dPI» 
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[78] to conclude that 


lim S(t,0.O = 

00 

exists pointwise in 0 and and is given by (6,71)« 

Remark 6.5*1 The filtering error covariance operator E(t,0,Q is 
monotonically increasing in t because we have zero initial estimation 
error (i^e*, the initial condition for the system is deterministic 
and knovm, taken to be zero in our formulation)* If we have a random 
initial condition, we may still be able to prove (although we have 
not done so) that the resulting E(t,0,5) converges to 2^(0, Q, since 
initial conditions are usually "forgotten” as t We may note 

that analogous situations also arise in the filtering problem for 
systems without delays, 

6, 6 Stochastic Control of Linear Delay Systems 
and Stability 

The preceding development in the stability of the optimal closed*- 
loop control system and the stability of the optimal linear filter 
suggests that by putting the two together, asymptotic stability of 
the closed-loop stochastic control system can be obtained* This is 
of course the case in linear stochastic system without delays. In 
this section, we examine the corresponding stochastic control system 
for delay systems and prove a similar result. 

We shall first discuss the separation theorem for delay systems 
as proved by Lindquist [56]. Again we consider the following linear 
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(6.72) 



stochastic delay system 

dx(t) =* lAx(t) + Bx(t-T) + Gu(t)]dt + M’dw^^Ct) 

x(0) = (j>(0) 0el-T,O] 


dz(t) = Cx(t)dt + Ndw^Ct) (6.73) 

standard Wiener processes, completely independent of each 
other, and independent of the initial function tj), which is a Gaussian 
process. The matrix N is assumed to be positive definite. The ob- 
jective is to choose a control law u in a suitable set of admissible 
controls such that the cost functional 

T 

Jiji(u>((>) = E r [x’(t)Qx(t) + u' (t)Ru(t) ]dt (6.74) 

*0 

is minimized, where T< 


Before we define the set of admissible controls, we establish 
some notation. By the variation of constants formula, we obtain 

-t -t -0 

z(t) = I C$(s,0)(j>(0)ds+ I I C§(s,a+T)B<j)(a)dads 

1) ’1) -T 


rt y.S 


-t -s 


JJ C*(s,a)Gu(0)dads + rr C$(s ,cr)M’dw(0)ds+Nw2(t) 


0 0 


0 0 


r“t y.S 




r r $(s,0)Gu(0)dOds 

*b *{) 


(6.75) 


i: 


J 

I 

! 


1 

• i 


Define the set consisting of the class of processes u(t) satisfying 
the following conditions: 


(i) 

(ii) 

(iii) 

(iv) 

We shall take U to be the set of admissible controls. The following 
o 

theorem has been proved by Lindquist [56]. 

Proposition 6.6.1 The problem of determining u e so as to minimize 
(6.74) has the following solution 

u*Ct) = -R"^G'K^(t)x(t|t)-R"^G' J K^(t,0)x(t+e|t)d0 (6.76) 

“T 

where K^(t) and K^(t,6) are the optimal gains for the deterministic 
optimal control problem (section 6.2), and xCsjt), t-x ^ s ^ t is the 
conditional expectation of x(s) given z(cf), 0 ^ a _< t (section 5.5). 

Remark 6.6.1 A few comments on the choice of U as the set of ad- 
o 

missible controls are in order. Let us define the set U to be the 

class of processes u(t) that are measurable with respect to 

a{z(s)s 0 £ s £ t} for each t, such that for each u e U, there exist 

2 

unique solutions to (6.72) and (6.73), and for which I E|u(t)| dt < «>. 

0 


u(t) is measurable with respect to c{s(s), 0 £ s £ t} 
for each t; 

For each u £ U , there exist unique solutions to (6.72) and 
o 

(6.73); 

rp 

p E|u(t)pdt < “ 

■Tl 

For each u £ U , a{z(s), 0 < s < t} = a{z (s), 0 < s < t}. 
o — — o 

In words, (iv) means that by using a control law u e 
we cannot gain, from the resulting outputs, more information 
about the system than that obtained by using no control at all. 
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I 


Clearly U C U. The basic difference between the sets U and U is 

that the requirement a{ 2 (s), 0 5. s ^ t} = a{z^(s), 0 s _< t} is 

not imposed on U. However, if we examine the proof of Proposition 

6*6.1 in [56], we see that the requirement of a{s(s), 0 £ s £ t} “ 

a{Zo(s)» ^ ® ^ admissible laws, is 

crucial- One would really like to prove that the separation theorem 

holds also for the set U (which is equivalent to showing = U) but 

the truth or falsehood of this is not known at present. We may note 

that in the separation theorem without delays [79 ] » the conditions 

(i) to (iv) satisfied by all u e are obtained by requiring the 

control laws to be Lipschitz functionals of the past observations - 

Even in the case without delays, it has not been established that 

U = U. 
o 

The expression for the optimal cost is given in 

Lemma 6.6.1 Corresponding to the optimal control (6.76), the optimal 
cost J* associated with the stochastic control problem (6.72) - (6.74) 
is given by 

trM'MK^(t)dt 

+ f tr{K^(t)GR"^G’K^(t)P^Ct)+ f (t, 0)GR~^G*K^ (t)Pj(t,0)d6 

0 ° -T 


J* = EV(x ) 
o 


f 


r 

“T 


K^(t)GR ^G’Kj^Ct; 


.0 /.O 


0)Pl(t 


, 9) dG^f 


“T 


t,0)GR“VK^(t,O' 


•p2Ct,€,0)d0d?}dt 


(6,77) 


- 162 - 


where V(x^) = x' (t)K^(t)x(t;+ J x* (t)K^(t,0)x(t+0)d0 

-T 


0 -0 


J x'(t+0)K’(t,0)x(t)d0f J J x’(t+ 0 )K 2 (t, 0 ,Qx(t+Od 0 d? 

“T -T -T 


Proof ; See Appendix C, 


(6.78) 


Remark 6.6.2 Lemma 6.6.1 shows that the optimal cost in the delay case 
has the same structure as the case without delays: there is a term due 
to initial conditions, a term due to the noise in the system dynamics, 
and a term due to the estimation error. 


We turn our attention now to the stochastic control system defined 
by using the steady state version of (6.76); 


u(t) = 


-R"^G'K X 

O 


1 /*o 

(t|t)-R"-^6' I K^(0)x(t+0[t)d0 


(6.79) 


where x(t[t) is generated by the steady state optimal filter. Heuristi- 
cally, this law is the optimal one for the functional 


E 



Ex'(t)Qx(t) + u' (t)Ru(t) ]dt 


(6.80) 


with observations started back at except that it would give rise to 
an infinite J*. One must therefore modify the control problem to be 
one of minimizing 

T 

lim r [x’(t)Qx(t) H- u* (t)Ru(t) ]dt} 
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or 


[x* (t)Qx(t) + u' (t)Ru(t) ]dt 


liin 

oo 


1 

T 



This introduces notions of ergodicity and invariant measures into the 
discussion. Even in the case without delays, the infinite problem has 
not really been completely resolved. We therefore content ourselves 
xd-th simply the result on asymptotic stability of the closed-loop 
system under the law (6.79). 


Theorem 6.6.1 Let Q = H'H. Assume (A,B,G) is stabilizable, (A,B,M') 
is controllable, (A,B,C) is detectable, and (A,B,H) is observable. 
Then the control law 

u(t) = -R"^G'K^x(t|t)-R"^G' f K^(0)x(t+elt)d0, 

“'-T 

where x(t[t) is generated by the steady state filter of Theorem 6.4.1, 
and K^, K^(9) are generated by the deterministic stationary control 
lax? of Theorem 6.2.1, gives rise to an asymptotically stable closed- 
loop system. 


Proof ; Stabilizability of (A,B,G) ensures that K^, K^(0), and 1(^(0,^) 
are x7ell-de fined. Observability of (A,B,H) then guarantees that 
solutions of the system 

—1 —1 
x(t) = (A-GR -^G'K^)x(t)+Bx(t-T) - | GR G’Kj^(0)x(t+6)d0 

(6.81) 

are asymptotically stable (see Theorem 6.2.1). Detectability of 
(A,B,C) ensures that the steady state filter is well-defined, and con- 
trollability of (A,B,M') guarantees that the steady state filter is 
asymptotically stable. The closed- loop system is defined by the 
coupled set of equations 


■: I 


■I 
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dx(t) = Ax(t)dt + Bx(t’-T)dt“GR ^G*K x(tjt) 

o 


- j GR G'K^(e)x(t+ 0 |t)d 0 + dxiT(t) ( 6 , 82 ) 


dx(t|t) = Ax(t|t)dt + Bx(t“T| t-x)dt 


+B 


J P^(-T,s-t)C' (NN’)“^[dz(s)-Cx(s|s)ds]dt 

t-T 


+P C'(NN') ^[dz(t) - Cx(t|t)dt] 


( 6 . 83 ) 


Expressing x(t|t) = x(t) - e(t|t)i xi;e get 


and 


dx(t) = (A-GR ^G'K )x(t)dt + Bx(t-T)dt 
o 


r GR ^G’K^( 0 )x(t+ 0 )d 0 + dw(t) 

“T 


+GR ^G’ 


1 

K^e(tjt)+ I GR G’Kj^( 0 )e(t+ 0 (t)d 0 ( 6 . 84 ) 

-T 


de(t|t) = [A“P C*(NN’) ^C]e(tjt)dt + Be(t-T| t-x)dt 


1 

+ I Bp2(-T,0)C’(NN') Ce(t+0lt+0)d9dt 

-T 


-1 -1 
+ dw(t)-P^C'R -^dv(t) - B P 2 (-t, 0 )C’R ^dQv(t+0)dt 


(6.85) 
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Since (6.85) is decoupled from ( 6 . 84)3 the stability properties of 
the closed-loop system are precisely those of (6.81) and the steady 
state optimal filter. Since both of these are asymptotically stable 
as a consequence of our assumptions, the closed-loop stochastic control 
system is asymptotically stable as well. 


Remark 6.6.3 T^hen the control law (6.79) is applied to the problem 
of minimizing 


J 

r 


lim “E 

'JM- CO 



[x' (t)Qx(t) + u’ (t)Ru(t) Idt 


Lemma 6.6.1 shows 


j = trM'MK 
r 0 


-1 -1 
+ tr{K GR G'K P + I KJ (0)GR -^G'K P’ (O)d0 
o ooj 1 ol^ 

-T 


,0 .0 


J K„0R-l0’K^(e)P^(6)d9+ JJ 


Kj^(e)GR 


-1 


G’Kj^C?)p2(C,0)d5de} 


-T -T 


This can be thought of as the cost rate associated with the control 
law (6.79). 


6.7 Filter Stability for Systems with Delays in 
The Observations 

In the previous sections, we have discussed the stability properties 
of the optimal filter and control system rather thoroughly for linear 
stochastic systems with a single delay in the system dynamics. It is 
relatively straightforward to extend the dxiality theorem of section 6.3 
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to cover situations with multiple delays in the state dynamics. 

Combining this with the results of Delfour, McCalla, and Mitter [32] 
on optimal control, it should not be difficult to extend our stability 
results to these systems. liJhat is more interesting is the case in 
xjhich there are delays in the observations. Since similar difficulties 
arise for distributed delays as well as point delays, we shall, for 
convenience, consider systems of the form 

dx(t) = [Ax(t) + Lx(t-T)]dt + dw(t) (6.86) 

x(9) =0 “X _< 0 0 

dz(t) = [C^x(t) + C^x(t-x)]dt + dv(t) (6.87) 

From the results of Chapter 5, xte knox'f that the optimal filter is given 
by the equations 

dx(t|t) = [Ax(t|t) + Bx(t-x|t)ldt 

+ [P^(t)C^R~^ + P2(t,0,-T)C^R"^] 

*[d 2 (t) - C^x(t|t)dt - Cj^x(t-Tl t)dt] 

( 6 . 88 ) 

x(t+0|t) = x(t+0lt+0)+ f [P 2 (s,t+ 0 -s,O)C^ + P2(s,t+9-s,-T)C[]R' 

*t+0 

•[dz(s) - C x(s|s)ds - CTx(s-x|s)ds] 
o' 1 ' 

(6.89) 

If xi7e appl> the same technique as that of section 6.4 to (6.88) - (6.89), 
we see that 
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dx(tlt) = [AxCtjt) + Bx(t-Tlt-T)]dt 


/ 


+ B I [p2(Sjt-T-s,0)C^ + P2(s,t-T-s,-T)C^]R 
t-T 


*[dz(s) - C^x(s|s)ds “ Cj^x(s-T I s)ds]dt 


+ tPo(t)C^ + P2(t,0,-T)q]R 


-1 


[dz(t) - C^x(t|t)dt - C^x(t-T| t)dt] 


(6.90) 


which is not a stochastic delay equation in x(t|t). Indeed, it is not 
possible to express x(t-Tjt) solely in terms of x(s|s), t - t s ^ t. 
Without this, the technique presented in section 6.4 cannot be applied. 


Is 


The situation is very similar in the dual control problem, which 
to minimize: 


J,j,(b,u) 


r [y' (t)Qy (t) + u' (t)Ru(t)]dt 
0 


subject to the constraint 


y(t) = -A’y(t) - B’y(t+r) - C^u(t) - Cj^u(t+r) 
y(T) « b 

y(s) = 0 s > T 

u(s) = 0 s > T 


Theorem 6.3.1 shows that the optimal estimate of b'x(T) for (6.86) - (6 
."T 

is given by 


J 

•'0 


^(s)dz(s), where u^(t) is the optimal control to the 


dual problem. 


. 87 ) 
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By the time reversal technique of section 6.4s we see that the 
above control problem is equivalent to an optimal control problem 
for systems ^^?ith delays in the control as well as in the state. It 
is knotm [80] that this problem is much more complex than the optimal 
control problem with delays in the state only. To understand the 
structure of the optimal filter when there are delays in the observa- 
tions, it is worthwhile to discuss briefly the control problem with 
delays in both the state and control. 


Consider the system 

x(t) = Ax(t) + Bx(t-T) + C u(t) -I- C-u(t“T) 

o 1 

x(9) = (f)C0) 0 c [-T,0] 

u(0) =0 0 E [~T,0) (6.91) 


The problem is to find a control u such that the cost functional 




,<i>) = r 

*^0 


[x' (t)Qx(t) + u’ (t)Ru(t) ]dt 


is minimized. The solution to this problem is given in [80] s 


u(t) = -R ^C^[P^(L,t,t)x(t)+ 


f 

+ P^(t,s+T,t)C2u(s)ds 


P^ (t , s+T , t ) Bx (s ) ds 


-Y(t)R~^C*[P (t+T,t,t)x(t)+ 
1 c 


/; 


r 

t-T 


P^Ct+T jS+T,t)Cj^u (s)ds] 


P (t+T,s+T,t)Bx(s)ds 
c 


(6.92) 
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where Y(t) = 


0 if t e [T-t,t] 


1 otherwise 


and P^(t,s,r) is a type of Fredholm kernel introduced by Manitius [71] 
and satisfies appropriate partial differential equations. 


The optimal control u(t) is not expressed solely in terms of x^, 
but also in terms of u(s), t-T ^ s < t. In order to express u(t) 
solely in terms of x, we can regard (6.92) as an Integral equation in 
u(t). In fact, if we write 

q(t) = -R ^C*[P (t,t,t)x(t)+ f P Ct,s-fX,t)Bx(s)ds] 
o c J ^ 


t-T 




-Y(t)R c;iP^(t4.T,t,t)x(t)+ I P^(t+T,s-bT,t)Bx(s)ds] 


r 

*t-T 


and 


N(t,s) = -[R Vp^(t,s+T,t)Cj^ + Y(t)R"^Cj^P^Ct+T,s+T,t)Cj^I 


(6.93) 


we get 


u(t) = q(t) + J" N(t,s)u(s)ds 
t-T 


(6.94) 


The properties of P^(t,s,r) [80] clearly implies that N(t,s) is a 
Volterra L^-kernel. Substituting (6.94) into itself, we obtain 
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u(t) = q(t) + / N(t,s)q(s)ds 


/: /: 


N(s,r)u(r)drds 


= q(t) + I N(t,s)q(s)ds+ 


J /‘t rinin(r+T,t) 

I I NCt,s)N(s,r)dsu(r)dr 

t-2T*qnax(t-T,r) 


By induction, it is easily seen that in fact u(t) satisfies a Volterra 
integral equation of the form 


u(t) = q(t) + I M(t,s)u(s)ds 


v7ith M(t,s) an Lg-kemel. The solution for the optimal control u in 
state feedback form can therefore be expressed as 


n(t) = q(t) + I R(t,s)q(s)ds 


dgF(t,s)x(s) 


where F(t,s) is an L2~kernel of bounded variation in s. Notice that 
in state feedback form, the entire past of the state trajectory must 
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There are two ways to study the stability of the closed-loop system. 
We can either study the system defined by (6.91) and (6.94) as system of 
coupled differential and Integral equations, or we can substitute (6.95) 
into (6.91) and study the resulting integrodifferential equation. In any 
case, the situation is considerably more complex than that without delays 
in the control. Stability properties of the optimal system are left as 
a subject for future research. 

One can see that a similar situation happens in the optimal filte;r 
for systems with delays in the observations. Defining 

W(t,s) = [P2(s,t-T-s,0)C^ + P 2 (s>t“T-s,-T)C^]P 

x(t-x[t) = x(t-T)t-T) + f W(t,s)[dz(s)-C^x(s[s)ds] 

“t-T 


-r 

t-T 


W(t,s)C,x(s-x| s)ds 


(6.96) 


One can view (6.96) as an integral equation in x(t-x[t). Again the 
kernel W(t,s)C^ is a deterministic L 2 -kemel. As before, we can solve 
x(t-x|t) in terms of x(s|s), 0 <_ s £ t. The stability of the optimal 
filter can thus be studied from either an integrodifferential equation 
or from a coupled set of differential and integral equations. In view 
of the duality theorem between estimation and control, the stability of 
the dual control system should be intimately related to the stability of 
the optimal filter, in the same sense as the connection discussed in 
section 6,4. 
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6.8 Perspective 


In this chapter, we have studied the stability of the optimal 
linear control system and the optiiual linear filter for systems with 
delays only in the state dynamics. Since the main results are inter- 
connected in a rather complicated way, it seems x^orthwhile to recapitu- 
late the various steps involved and put our results in perspective. 

We first shoved that under the conditions of stabilizability and 
observability, the inf ini te- time optimal linear control system with 
quadratic cost is asymptotically stable- This result not only extends 
previous work by other investigators, but also shows the important 
role played by system-theoretic concepts. 

In section 6.3, we established a duality theorem which relates 
the solution of the optimal linear filtering problem to the solution 
of a dual optimal control problem. This result was motivated by the 
work of Lindquist [75 la and Hitter and Vinter [67], and paved the way 
for exploiting the results of section 6.2 in studying the stability of 
the optimal linear filter. 

Theorem 6.4.1 in section 6.4 contains the central results of 
this chapter, and combines the results of Chapter 2 with those of 
sections 6.2 and 6.3. First, we used the duality theorem to give 
the precise formulas relating the optimal gains and the fundamental 
matrix of the optimal filter to those of the corresponding dual 
optimal control system. This is the content of Lemmas 6.4.1 through 
6.4.4. Next, we used the duality results of Chapter 2 to assert that 
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detectability and controllability of the original system corresponds 
to stabilizability and obseirvability of the dual system. Thus, under 
the assumptions of detectability and controllability of the original 
system, we were able to invoke the stability result of section 6.2 to 
conclude asymptotic stability of the dual optimal control system. 

Finally, the formulas relating the optimal filter and the dual optimal 
control system allowed us to conclude asymptotic stability of the 
optimal filter from that of the dual control system. Once these 
results were established, it was straightforward to prove, in section 
6.5, that the closed-loop stochastic control system is also asymptotically 
stable. While the analogs of these results for ordinary differential 
systems are well-knoxTO, they are new for linear delay systems, and 
constitute the first complete extension of the finite dimensional 
linear-quadratic-Gaussian theory. Furthermore, they bring into focus 
the importance of the structural properties and duality relations in 
control and estimation problems for linear delay systems. liJhile our 
results deal only with a special class of delay systems, we feel that 
the ideas and techniques will be fundamental to the development of a 
complete linear-quadr.’ci c-Gaussian theory for more general delay 
systems, tte hope that thr work reported in this thesis will serve 
as a stimulus to the development of such a theory. 


CHAPTER 7 


CONCLUSIONS AND SUGGESTIONS FOR FUTURE RESEARCH 

In this thesis, we have studied two areas in the theory of delay 
systems: structural properties and their applications to control in 
linear time- invariant delay systems, and optimal linear and nonlinear 
filtering for stochastic delay systems* We summariEe our findings 
here and indicate what we feel are the main contributions of the 
thesis* 

In Chapter 2, t^e discussed thj various notions of controllability, 
st'^.bilizability, observability, and detectability in connection x^ith 
linear delay systems* New definitions of controllability and obser- 
vability were given and some duality results established. ^-Jhile we 
have not conducted an in-depth study of these concepts, the ideas 
presented have proved to be very useful connection x^ith linear 
optimal control x^ith quadratic cost and optimal linear filtering. 

In Chapter 3, we sfidied the con--'" t of pointx^ise degeneracy. 

We completely characterized 3 dimensional systems x^hich can be made 
polntwise degenerate by delay feedback, and gave sufficient con- 
ditions for this to be possible in higher dimensional systems. The 
conditions obtained are felt to be much simpler and more intuitive 
than previous results obtained by other investigators. 

In Chapter 4, x^e applied the pointwise degeneracy property to 

the control of linear systems. A complete solution to the minimum 


- 175 - 


time output zeroing problem by delay feedback was given for 3 dimen- 
sional linear systems, while sufficient conditions were given for 
the solvability of the problem in higher dimensions* The solvability 
conditions are again very simple, although the actual construction of 
the feedback controllers is rather complicated. Sensitivity of the 
control system under perturbations of its parameters was also studied 
and it was shc«m that "small’* perturbations give rise to "small" 
degradation in the system performance. Some examples were then given 
to illustrate the theory. All these results are new and show that 
the concept of delay feedback has potential applications in linear 
system theory which are worth exploring further. 

We began the study of the optimal filtering problem in Chapter 
5, The general no:n:linear filtering problem was first studied, and 
a representation theorem for conditional moment functionals derived. 
The form of the representation is new even when specialized to 
ordinary differential systems, and can be thought of as solving the 
nonlinear filtering problem abstractly. We then showed that if the 
moment functionals were suitably "smooth," stochastic differential 
equations could be derived. When these results were specialized to 
the linear case, we obtained a complete solution to the optimal linear 
filtering problem, generalizing previously knorm results* 

In Chapter 6, we studied the stability properties of the optimal 
linear filter. The special case where the observations do not in- 
volve any delay terms was considered. We showed that there is a 
duality relation between optimal control with quadratic cost and 
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optimal linear filtering. Under the hypothesis of stahilizability 
and observability a we showed that the optimal closed-loop system 
for the infinite time control problem is asymptotically stable* 

Under the hypothesis of detectability and controllability 5 we 
showed that the optimal linear filter is asymptotically stable* 

Finally, the closed-loop stochastic control system using the deter- 
ministic optimal control law and the optimal filter estimate x-^as 
also shown to be asymptotically stable* This we feel is the first 
complete extension of the familiar linear-quadratic-Gaussian theory 
for ordinary differential systems* On the one hand, almost complete 
analogs of the finite dimensional results have been obtained: the 
linear feedback nature of the solutions for the optimal control and 
filtering problems, the Riccati-type partial differential equations, 
the duality between control and filtering, and the stability results 
based on system-theoretic concepts* On the other hand, the additional 
complications due to the presence of delays are also clearly demon- 
strated by the form of the solutions. These similarities and differ- 
ences between the finite dimensional theory and the theory developed 
in this work give a lot of insight into the behavior of linear delay 
systems* 

The results obtained in this th isis suggest that further in- 
vestigations in the following topics would be fruitful. 

1. Study in greater depth the relations between the various 
notions of controllability, stabilizability, observability, 
and detectability for delay systems* In particular, algebraic 





conditions for checking these properties would be very 
useful. In addition to the notions discussed in Chapter 
2, there are other algebraic formulations of the notions 
of controllability and observability [42], [43]. It would 
be very worthwhile to relate these various approaches and 
obtain a unified theory* This is very much an open area 
of research* 

2. Extend the applications of the pointwise degeneracy property 
to the construction of delay observers* We have given an 
example of a "deadbeat" observer. The general theory, 
however, is lacking, and it would be interesting to see 
what precise conditions are required- It is suspected that 
this problem will be related to the properties of the adjoint 
system* 

3. In studying the nonlinear filtering problem, we have shown 
that in order to derive stochastic differential equations, 
moment functionals must be in the domain of the infinitesimal 
generator of the Markov process x^. The characterization of 
this operator is important in its own right as well as in 
applying the results of Chapter 5* It is hoped that a de- 
tailed characterization of this operator will pave the way 
for useful approximations to the optimal nonlinear filter. 

4* Extend the stability results of Chapter 6 to cover more 
general models, for example, those with delays in the 
observations. As discussed in Chapter 6, this will be 
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intimately related to the control problem with delays in 
the control* It is not clear what techniques are suitable 
for this problem. In fact, it is not clear whether the 
controllability and observability concepts which have 
proved fruitful need to be reformulated or not* Since 
many practical systems have delays in the observations, 
this is an important practical as well as theoretical 
problem. 
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APPENDIX A 


PROOF OF LEMMAS 4.1 AND 4,2 

We refer the reader to Chapter 4 for the statement of these 
lemmas . 

Proof of Lemma 4.1 ; The extrema of the function f(X) = (ctj^+a 2 A.+a 2 X^)e^^ 


occur at 


A = 


2a^T 


-(a^T+Za^) 


n 


Since > 0, there are two real extrema for f(X)* Also, the 

2 

zeros of f(A) occur at the zeros of a^+a^X+a^X s and these are at 


-“2 

2al 


Again, since > 0, these give two real zeros of f(A). Let 


®1 




2a, 


®2 


-a^ a2^-4a^a. 


2a, 


Let us perturb A^ by some number e and let 




= E(a,e+2s,a„+a_) 

J 1 j 2 
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Hence 




n^(s2) == eCa^e-^ya^ -4a^a^) 


This yields 


n^(si) > 0 


if £ > 0, £r.d ea^ > 




if e > Oj and ea. 


/fat^ “4a^ct^ 


if £ < 0» and ea^ < 


0^2 


if e < 0, and ca^ > 




Similarly, 


\(S2> > 0 


if e >0, and £a^ 


if e > 0, and £0t^ 


if e < 0, and ea^ <^/ . "4a^a^ 


if E < 0, and Ea^ >yj / 


Let us now examine the graph of the function f(A). There are 
eight cases, depending on the signs of the 

Case I: > 0, > 0, > 0. In this case and S 2 both negative 

If £ > 0, r| (s^ ) > 0, and if 
e l 




< £ < 0, ng.(s^) < 0. 


A little thought now shows that the graph of f(A) is of the form 



It Is obvious that for any finite real 3 or 82* there exists a 

Yg such that the line Yg(X-3) intersects f(X) at three real distinct 
values of X, none of which equals s^^ or 82* There also exists a 
constant w(o) > 0 in this case) such that the line A=(i) intersects f(X) 
at three real distinct points. 


For the other seven cases, entirely similar arguments show that 
f(X) must have one positive maximum and one negative minimum, and 
hence the required straight lines exist. We shall omit the details. 
The lemma is now proved. 


Proof of Lemma 4 . 2 t Since C is of full rank, we may assume, without 

,th 


loss of generality, that q^c^ = 0, where c^ is the i column of 

C. Since (A,C) is controllable, we have the following possibilities: 


(i) {cj^, Acj^j C2} are linearly independent 


(ii) {cj^, C2, AC2} are linearly independent 


or 


Case (i) : {c^, Ac^^, C 2 } linearly independent. 

Construct a matrix as follot^s: 

= [0 62 0][Cj^ 

where 62 is the second natural Euclidean basis vector in 2 dimensions 
and ^ is the null vector. Then 

(A + CKj^)c^ = ACj^ 

(A + CKj^)^ = A^c^ + C2 

and (A^jCj^) is controllable, where A^^ = (A + CK^) . 

3 2 

Let the characteristic polynomial of A. be p(s) = s +p„s 4p s+p 

JL ^ J. O 

We can then x-jrite 

C 2 “ '^l''‘^2^l‘^l'^^l‘^l^ ^2 ^^l‘^l*'^2‘^l^ ^3°1 


for some 3^, 32 » ^ 3 * independence of c^^, Ac^^, and C 2 implies 

3j^ i‘ 0. The transformation 


P = tA^ A^ 


Cl Cil 


10 0 
?2 1 0 

Pi pj 1 


gives P ^A^P in companion form and 



C = 


0 

0 

1 
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Let q*P - * Then 

2 

= q' (A c^+c^) + p2q'Ac^+Pj^tt2 
“2 = 'i'^‘^l+P2“3 

2 ®2 

If a^ « ^^1^3 ^ 1 ^ ^ zero of n(^) = 

are done. If not* take a matrix F of the form 



then P"^(A+CK^)P + P~^CF 

= P"^(A+CK^+CFP“^)P = p"^A2P 




^1 " ^ *-^2 ^ 2 ^^! 


1 


0 



Then P A^PP^ - P 2 A^P 2 


0 

0 


1 

0 


0 

1 


-Po -Pi -P2 


Furthermore P^^ P C = P 2 C 




’0 d 


12 


0 d 

Li d 


22 


32 -J 


’ some d^2. <i22> 


Now A 2 Cj^ = (A+CKj^) c^+CFP 


Ac^+C 




“ ‘^'^1 ^ 2*^1 


2 2 2 
Similarly A 2 = A c^+C2~f2^'^l‘^^^2 -P2^2'*'^l^ '^l 


Choose f^, and £2 to satisfy the properties 

2 

(a) -4a^a2 > 0, where 


= as = 


“2 ^ 'l'^2‘^l’’’P2“3 “ “2 


and aj = q*A2^Cj^+P2q*A2C^+pj^a2 


and 


= q’ (A2c^+C2)+P2q'Ac^+(f2^-f2-P2f2-^l+2f3_)a3 


22 2 
(b) — is not a ^ero of r\' (X) = aj^+ot2X+ot2X , 
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A little algebra shows that in fact *^22 * ^2’ 

^32 “ 


Some reflection now shows that appropriate choices for f^, f^^, 
and f 2 exist . This completes the consideration of case (i) . 

Case (ii) : ^^ 2 ^ AC 2 } linearly independent. 

We construct a matrix K 2 as follows 
S = [®2 C2 AC2] ^ 

2 2 

Let = A+CK^. Then = Ac^+c^ ^3 ^1 ” c^+Ac^^^CK^Ac^^ * 

3 2 

Let the characteristic polynomial of A^ be p(s) = s +P 2 ^ 

We can write 

C2 = 61 EA 3 Cj^+P2A2C^+p^c^]+e2[A3C2^+P2Cj^]+e2<=^l 

$2 is clearly nonzero. If 3^ 0> we carry out the constructions in 

exactly the same fashion as in case (i) . This shows that we can again 

— 1 —1 

find appropriate matrices P and K such that P (A+CK)P, q'P^ and P C 
have the required structure. The proof of the lemma is completed. 
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APPENDIX B 


DERIVATION OP EQUATIONS (5.53) - (5.55) 

To derive the appropriate equations for P(t,0,Q, we have to 
consider three cases: 

(i) 0 < t < T, -T < 0 < 0, -T < C < 0 

(ii) 0<t<T, -T<0<O, ?=0 

(this also covers the case 0^t<^Tj 9 = 0, 
since clearly P(t,6,C) = P’(t,5,0)). 

(iii) 0£t<T, 0 = ?=O. 

Since the derivations are somewhat tedious, we shall give details only 
for the equation satisfied by P(t,6,5) for 0 ^ t ^ T, -T ^ 9 < 0, 

-T _< ? < 0. To avoid notational complications (since P(t,9,^) is a 
matrix, to use Theorem 5.3.1 requires the calculation of P^^(t,0,^), 1 
i,j 5. n) , xie will assume everything is scalar. For convenience we 
shall write e (x ,s) = c(x ,s) - c(x ,s) 

C 5 5 3 

We appeal to Theorem 5.3.1 and (5.50) to write 

P(t,0,5) = E^{x(t+0)x(t+?)}-x(t+0|t)x(t+Cjt) 

= E {x(t+0)x(t+?)}+ r E®{E [x(t+9)x(t+^)lx ]e (x ,s)]} 
o 1 ^ s c s 

•'0 

•R~^(s)dv(s) 
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-[x(t+0|t+e)+ r E®{x(t+0)e^(Xg,s)}R"^(s)dv(s)] [5^(t+C|t+5) 
Jt+0 


f ' 

^t +4 


+ { E®{x(t+^)e (x ,s)}R ^(s)dv(s)] 

c s 


= E [x(t+0)x(t+C)l “ x(t+0lt+0)x(t+Clt+a 




+ I E®{E [x(t+0)x(t+C){x ]e (x ,s)}R ^(s)dv(s) 
■ o s c s 


f ' 

JtH 


-x(t+0|t+0) ( E®{x(t+5)e (x ,s)}R ^(s)dv(s) 


C' S' 


-x(t+Clt+^) j E®{x(t+0)e^(Xg,s)}R ^(s)dV(s) 


J 

-'t+0 

t t 

J E^{x(t+0)e (x ,s)}R ^(s)dv(s) (* E®{x(t+^)e (x ,s)}R ^(s)d\>(s) 

C 5 I C S 

t+0 t+C 


(B.l) 


It is readily seen from (B.l) that P(t,6,€) is continuous in (t,0,^). 
Similarly for z in (0,t) such that -r£0 + £^O, -T£^ + e<_0, 
P(t-e, 0+e, C+e) 

= E {x(t+0)x(tH)}-x(t+0!t+0)x(t4-C|t+O 


J rt-e - 

E®{E^[x(t+0)x(t+C) |Xg]e^(Xg,s)}R ’‘■(s)dv(s) 

0 


^t-E ^ 

- S(t+0jt+0) I E®{x(t+^)e^(Xg,s)}R ■’■(s)dv(s) 

Jt+% 
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^t-e 

1 

Jt+0 


t-e . 

-x(t+dt+?) I E®{x(t+0)e^(x^,s)}R"^(s)dv(s) 

C 5 


-t-E _t-e 

I E®{x(t+0)e (x ,s)}R ■^(s)dV(s) I E^{x(t+?)e (: 
I c s I c 

j t +0 j t+€ 


Hence P(t,0,^) - P(t"E, 0 +Ej ^+e ) 

ft 


J ' 

^t-e 


E®{e [x(t+ 6 )x(t+ 5 ) |x ]e (x ,s)}R ^(s)dv(s) 
o s c s 


I ^ 

^ t-e 


-x(t+9|t+0) I E^{x(t+C)e^(x^,s)}R ^(s)dv(s) 


-x(t+?lt+S) I E®{x(t+0)e^(x^,s)}R ^(s)dv(s) 


J t-e 


/ t -t 

E^{x(t+0)e (x js)}ll (s)dV(s) | E^{xCt*h?)e (x 
C S I c s 

t4*0 •/ t+4 


+ I E®{x(t+0)e (x ,s)}R ^(s)dv(s) I E®{xCt-f^)e ( 
I C s I c 

^ t+9 ^ t-hE 


.t-e 

i 


The last two terms of (B.2) can be written as 


f 

J t“*e 


E^{x(t+0)e^(x^,s) }R 


■^(s)dv(s) f 

t“l-^ 


E®{x(t+C)e, 


- I E®{x(t+0)e (x ,s)}R (s)dv(s) I E®{x(t+^)e ( 
I c s I c 

t+0 J t-e 


: ,s)}R ^Cs)dv(s) 

s 


,s)}r ^(s)dV(s) 

X ,s)}R ^(s)dv(s) 
s 

(B.2) 

(Xg,s)}R ^Cs)dv(s) 

X ,s)}R ^(s)dv)(s) 
s 
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I E {x(t+0)e^(Xg,s)}R -^(s)dv(s) I E''{x(t+^)e^(Xg,s)}R (s)dv(s) 

Jt-e *^t-e 

i t 

E^{x(t+9)e (x as)}R (s)dv(s) I E {x(t rC)e (x »s)}R (s)dv(s) 

C s I C £> 

t-£ •^t+E 


r E®{x(t+9)e Cx ,s)}R~^(s)dv(s) f E®{x(t+Oe (x >s)}R ^(s)dv(s) 

I £ 5 1 C S 

Jt+9 •'t-e 

(B.3) 


Substituting (B.3) into (B.2) and using properties of Ito integrals. 


P(t,0,O - P(t-e, &fE, C+e) 


C' 


[x(t+6)x(t+E) ! X (x ,s)}R (s)dv(s) 

5 C. S 


-x(t+0|t-e) r E®{x(t+C)e^(Xg,s)}R~-^(s)dv(s) 

Jt-e 


-x(t+^|t-e) f E®{x(t+0)e (x ,s)}R ‘^(s)dv(s) 
1 c s 

Jt-£ 


E®{x(t+0)e (x^,s)}r“ 
c s 


{x (t+4) (x^ 9 u) }R (u) dv (u) dv (s ) 


r E‘^{x(t+0)e (x ,u)}R“^(u)E®{x(t+5)e„(x^,s)}R~^(s)dv(u)dv(s) 



E®{ 

t-e 

r E^E 

*^t-£ 


x(t+0)e (x ,s)}R ^(s)E®{x(t+^)e (x ,s)}ds 
c s c s 


{E^[x(t+6)x(t+C) iXg]ep(3tg>s)}R ^(s)dv(s) 


-r 

-'t-e 


x(t+?l s)E®{x(t+0)e (x ,s)}R ^(s)dv(s) 
c s 


-r^ 

•^t-e 


x(t+0{s)E®{xCt+5)e (x ,s) }R ^(s)dV(s) 
c s 


f 

•'t-e 


E^{x(t+0)e (x ,s)}R ^(s)E^ {x(t+C)e (x ,s)}ds 
c s c s 


(B.4) 


Since for each 0e[-T,O], x(t+0) given is Gaussian, the first three 
terms of (B.4) add to zero, being the third central moment of a 
Gaussian process. Hence 


P(t,e,C) - ?(t-£, 0 +e, 5+e) 


I E^{x(t-!'l 


9)e (x ,s)}R ^(s)E^{x(t+€)e (x ,s)}ds (B.5) 
c s c s 


Note that there are no random terms in (B.5). Since (B.5) holds for 

arbitrary e in (0 ,t) with -x £ 0 + e 0, -x £ C + £ £ 0, we can 
1 

divide by — and let £ go to 0. If we let CT be the unit vector in 
the direction (1, -1, -1), then the directional derivative, P|j(t,9,C) 
of P(t,0,?) in the direction O is given by 
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»^Pjy(t,e,5) = -E^{x(t+0)e^(x^,t)}R"^(t)E^{x(tH)e^(x^,t)} (B.6) 


Equation (B*6) is the desired equation for PCt^GaC), 0 _< t _< 

-T ^ 0 < Oj -T _< ? < 0* The same technique can be applied to the 
derivation of the equation for F(ts0jO), ^ 0 < 0, and 

that for P(t,0,0). We only sketch the steps, omitting the details. 

By Theorem 5. 3 .1, 

J rt 

E^{E rx(t4-0)x(t) |x ]e (x ,s)}R (s)dv(s) 
o s c s 

0 

-xCt+6|t)x(t jt) (B-7) 


(B.7) implies that PCt,0jO) is continuous in (t,0). Some calculations 
yield, for e in (0 ,t) such that -T £ 0 + e < 0, P(t,0,O) - P(t~0, 0+e, 0) 


= r E®{x(t+0)a(x ,s)}ds + r 

its ' Jt-. 

-x(t+0| t)x(t| t)+x(t+0 1 t-e)x(t-e| t-E) 

The last two terms of (B.8) can be evaluated to give 


E®{x(t+0)x(s)e (x ,s)}R ^(s)dv(s) 
c s 


(B.8) 


-x(t+0| t)x(t I t)+x(t+0 I t-£)x(t-£ j t-£) 


•'t-e 


=-r 

•'t-e 


“1. 


x(t-t-0 1 s)a(x^ ,s)ds“ I x(tT-0|s)E {x(s)e^(x^ 5 s)}R (s)dv(s) 


-i: 


E {x(t+0)x(s I s)e (x ,s)}R (s)dv(s) 
' c s 


E‘’{x(t+0)e^(x^,s)}R ^Cs)E®{x(s)e^(x^,s)}ds 


(B.9) 
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Substituting (B.9) into (B.8) and using again the Gaussian property, 
we see that 


P(t,6,0) - P(t-e, 6+e,0) 

= f E®{x(t+0)e (x ,s)}ds- f E®{x(t-}-0)e^(x^,s)}R"^(s)‘ 

I as I c s 

•'t-e *x-e 

•E®{x(s)e (x ,s)}ds (B.IO) 

C S 

Again there are no random terms so that P(t 50 ,O) has a directional 
derivative in the (1,-1) direction* Denoting the unit vector in the 
(1,-1) direction by rii we obtain 

^ P^(t,e,0) = E^{x(t+6)e^(x^,t)} 

-E^{x(t+B)e (x ,t)}R ^(t)E^{x(t)e (x ,t)} 

^ ^ ^ ^ (B.U) 

Finally, similar calculations (or using (5.42)) show that P(t,0,0) 
has a derivative and satisfies 

”P(t,0,0) = E'"{x(t)e^(x^,t)}+ E^{e^(x^,t)x(t)} 

-E^{x(t)e^(x^,t)}R“^(t)E{e^(x^,t)x(t)} + Q(t) 

(B.12) 

In the vector case, equations identical to those of (B-6), (B.ll), and 
(B.12) hold for P(t,6,5)» P(t,6,0), and PCt^OjO). This completes the 
derivation of equations (5*53) - (5.55). The initial conditxons (5.56) 
follow immediately from the properties of conditional expectations. 




APPENDIX C 


PROOF OF LEMMA 6.6.1 


We apply the Ito differential rule to the function V (x^) defined 
in (6.78). We calculate the first and second terms to illustrate the 
computations involved. 


dx’(t)K^(t)x(t) = [dx*(t))K^(t)x(t)dt 

+x* (t) [dK^ (t) ]x(t) dt+x' (t)K^ (t) [dx(t) ]dt+trM'MK^ (t)dt 

= x'(t-T)B’K (t)x(t)dt+u* (t)6’K (t)x(t)dt 
o o 

+dw' (t)K^(t)x(t)dt+x* (t)K^(t)Bx(t-T)dt-l-x* (t)K^(t)Gu(t)dt 


+x* (t)K^(t)dw(t)“x' Ct)QxCt)dtd-s' (t)K^(t)GR ^G'K^(t)x(t)dt 


-x' (t)K' (t,0)x(t)dt-x’ (t)K- (t,0)x(t)dt+trM'MK (t)dt 

JL ± Q 


f*' 


d^ I x' (t)K^(t,9)x(t+6)d0 


d^[x'(t) f K^(t,a-t)x(0)da] 
•'t-T 


r'(t)}J K^(t,( 


{[x’ (t)A’+x* (t-x)B*+u' (t)G’ ]dt+dw' (t)} I K^(t,9)x(t+9)d0 

'-T 

+x* (t)K^(t,0)x(t)dt-x* (t)Kj^(t,-T)x(t-x)dt 


+x’ (t) r d^K^(t,a-t)x(0)dadt 
Jt-x 


= [x' (t)A'+x* (t-T)B 


+dw 


r° 

'+u'Ct)GM K^(t, 
*^-T 


6)x(t+0)d0dt 


t,0)x(t+0)d0+x' (t)Kj^(t,0)x(t)dt 


’(t) I 

*^-T 

■X* (t)Kj^(t,-T)x(t-T)dt+x’ Ct) r [(1^ - -~j K^Ct,0)]x(t+6)d0dt 

T 


Similar calculations on the last two terms on the right hand side of 
(6.78) yields the following expression 

(t)dt+dw' (t)K (t)xCt)4-x* (t)K (t)dw(t) 

Z X o o 

+trM’MK^(t)dt+dw' (t) I K^(t,0)x(t+0)d9 


j .. 

.T 


(t+0)Kj^(t,0)d6dw(t) 


-X* (t)Qx(t)dt-u' (t)Ru(t)dt 


where ^1^*"^ ~ [u(t)+R ^G’K^(t)x(t)+ 


f 


(C.l) 


K^(t,0)x(t+0)d0]' 


'R[u(t)+R"^G’K^ Ct)x(t)+ 


r E-^G' 

tJ .jr 


K^(t,?)x(t+?)dU 
(C. 2) 


Using the boundary conditions at T for K^(t)j Kj^(t,0) , and K2(tj0,Oj 
we see that V(x^) « 0. Therefore 
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j Ix’ 

•^0 


(t)Qx(t)+u* (t)Ru(t)]dt 


= V(x^) + f V^(t)dt + r trM’MK^(t)dt 
•'O *^0 


+ 2 


f. 


(t)K^Ct)dw(t) + 2 


/T " 

Jq J~r 


(t+0)Kj^(t,0)d0dw(t) 


Taking expectations, we get 
-»T 


E r [x‘ 

jf\ 


E I [x' (t)Qx(t)+u' Ct)Ru(t) ]dt 
^0 


= EV(x^) -5- E j V^(t)dt + I trM'MK^(t)dt 


; f V (t)dt + f 

■^0 *^0 

Now E J* V^(t)dt = j EV^ 


(C.3) 


(t)dt 


E{E[V^(t)lz^]}dt 

0 

using Fubini’s theorem and properties of conditional expectations. 
Substituting the control law in (6.76) into (C.2) we get that 

E[Vj^(t)lz*^] = E{[R"^G’K^(t)e(t|t)+ f R‘^G'K^(t ,0)e(t+0 1 t)d03 ' 

•R[R"^G*K^(t)e(t|t)+ f° R~^G’K^Ct,0)e(t+ejt)d03lz^} 
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which is precisely 


tr{K^(t)GR"^G’K^(t)P^(t)+r Kj^(t,0)GR ^G’K^ (t)Pj^(t, 0)d0 

•'-T 


£-• 


(t ) GR"^G ' (t , 0)P^ (t , 0) d0 


O 0 

K^(t,0)GR ■^G'K^(t,PP2(t,C,0)d0de> 

T 


(C.4; 


I 

E{V^(t)|z } is now seen to be a deterministic function and hence 
equal to EV^(t). Substituting (C.4) into (C.3) yields the conclusion 
of the leirana. 
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